I added print(id(package)) in Lib/test/test_import/data/package/__init__.py: when the bug occurs on my Windows VM, I see two print with two different identifiers. Moreover, when the bug occurs, first id(sys.modules['package']) is the first identifier, and then package is replaced with the new 'package' package in sys.modules (new identifier).

I don't understand something: sys.modules isn't supposed to be protected by a lock? I checked if _installed_safely.__enter__() if the _imp is held: it isn't held when _bootstrap._setup() is called, nor when _imp._install_external_importers() is called. But even if I patch these two functions to acquire the lock, Python startup fails on importing a codec module, because the _imp lock is not held in _installed_safely.__enter__(). So maybe sys.modules dict is supposed to be protected by a different lock, maybe indirectly?

I hack _bootstrap._find_and_load_unlocked() to sleep 10 ms if name == 'package'. With this change, test_concurrency() fails immediatly on my Linux box.

* Apply attached importlib_sleep.patch (written for the current master branch)
* Recompile: make regen-importlib && make
* Run test_concurrency() just once: ./python -m test -m test_concurrency -v test_import
* You get the bug

By the way, importlib_sleep.patch changes test_concurrency() to run the test exactly once (it changes to loop to a single iteration, instead of 10).

Is it normal that a sleep makes import failing? Imports are not supposed to be atomic?

--

I tried to similar hack before the commit b4baacee1adc06edbe30ac7574d17a8cd168e2e0:

* add a sleep in importlib
* copy test_concurrency() from master
* Run test_concurrency()
* The test pass

While I'm not sure of my hack, it *seems* like sys.modules was somehow atomic before this commit?