This issue tracker has been migrated to GitHub, and is currently read-only.
For more information, see the GitHub FAQs in the Python's Developer Guide.

Author steven.daprano
Recipients George3d6, brett.cannon, eric.snow, ncoghlan, steven.daprano
Date 2021-08-15.02:20:57
SpamBayes Score -1.0
Marked as misclassified Yes
Message-id <1628994058.29.0.562007528827.issue44916@roundup.psfhosted.org>
In-reply-to
Content
The importing from multiple threads is possibly also non-deterministic, but I'm not an expert on the importlib module.

It looks to me like a another plausible source of random/arbitrary behaviour could be:


1. Within a single process, you have two threads running in non-deterministic order. So you could have, let's say:

- thread 1 imports M from file m1.py
- thread 2 tries to import M, and the import system sees that
  M is cached in sys.modules and uses that instead.


So even though the two threads are writing to different source files, they both call the module M, which means that you can have two threads stomping on each other's toes trying to import different classes C from different modules both called M.

I don't think this is supported at all. I'm not really qualified to rule out a bug in the importlib functions but to me it surely looks like a case of "don't do that".

Remember that sys.modules is cached globally per-process, so once you start pickling and unpickling your M.C instances, its a lottery which one you will get; furthermore, if you have instances:

    x = M.C()
    replace module M with a new module M
    y = M.C()

only y is using the new definition of C from the new module M, instance x is still using the original class with its original methods.

I'll leave it to Brett, Nick or Eric to confirm that there's nothing to fix in importlib, but my advice is to avoid using dynamically created modules **all with the same name** that touch the file system from multiple threads in multiple processes at the same time. There is far too many opportunities for non-deterministic behaviour to mess up your expectations.
History
Date User Action Args
2021-08-15 02:20:58steven.dapranosetrecipients: + steven.daprano, brett.cannon, ncoghlan, eric.snow, George3d6
2021-08-15 02:20:58steven.dapranosetmessageid: <1628994058.29.0.562007528827.issue44916@roundup.psfhosted.org>
2021-08-15 02:20:58steven.dapranolinkissue44916 messages
2021-08-15 02:20:57steven.dapranocreate