Author ncoghlan
Recipients brett.cannon, eric.snow, ncoghlan, serhiy.storchaka
Date 2017-07-08.07:19:27
SpamBayes Score -1.0
Marked as misclassified Yes
Message-id <1499498368.06.0.664954561474.issue30876@psf.upfronthosting.co.za>
In-reply-to
Content
If there are intermittent concurrent problems associated with this behaviour, I think that may be a sign that the current management of the per-module import locks is inadequate, since it isn't adequately accounting for direct manipulation of sys.modules in user code.

Fully resolving that would probably mean ensuring that:

1. For submodule imports, we always acquire the parent module locks in a consistent order, and then hang onto them until the submodule import is complete rather than letting them go as soon as the parent module import is complete
2. We move the current sys.modules to sys._modules, and make sys.modules a proxy mapping that acquires the relevant import locks in the same consistent order for set and delete operations before mutating sys._modules

If we did that, then we should consistently get one of the following two orders:

1. Thread A imports package, package.module1, package.module2; Thread B then unloads (and maybe reloads) package; or
2. Thread B then unloads (and maybe reloads) package; Thread A imports package, package.module1, package.module2

By contrast, at the moment there's a window where the following can happen:

- Thread A finishes importing "package" and starts importing "package.module1"
- Thread B unloads "package"
- Thread A hits SystemError through no fault of its own when it hits the "from . import module2" line

That said, formulating the problem that way does suggest another potential resolution: in this scenario, we could treat "from . import module2" *exactly* the same way we would handle "import package.module2", which would be to just import "package" again, rather than complaining that it "should" already be imported.

In addition to being vastly simpler to implement, that approach would have the virtue of also fixing the "del sys.modules(__package__)" case, not just the potential race condition.
History
Date User Action Args
2017-07-08 07:19:28ncoghlansetrecipients: + ncoghlan, brett.cannon, eric.snow, serhiy.storchaka
2017-07-08 07:19:28ncoghlansetmessageid: <1499498368.06.0.664954561474.issue30876@psf.upfronthosting.co.za>
2017-07-08 07:19:28ncoghlanlinkissue30876 messages
2017-07-08 07:19:27ncoghlancreate