Message298064
I added print(id(package)) in Lib/test/test_import/data/package/__init__.py: when the bug occurs on my Windows VM, I see two print with two different identifiers. Moreover, when the bug occurs, first id(sys.modules['package']) is the first identifier, and then package is replaced with the new 'package' package in sys.modules (new identifier).
I don't understand something: sys.modules isn't supposed to be protected by a lock? I checked if _installed_safely.__enter__() if the _imp is held: it isn't held when _bootstrap._setup() is called, nor when _imp._install_external_importers() is called. But even if I patch these two functions to acquire the lock, Python startup fails on importing a codec module, because the _imp lock is not held in _installed_safely.__enter__(). So maybe sys.modules dict is supposed to be protected by a different lock, maybe indirectly?
I hack _bootstrap._find_and_load_unlocked() to sleep 10 ms if name == 'package'. With this change, test_concurrency() fails immediatly on my Linux box.
* Apply attached importlib_sleep.patch (written for the current master branch)
* Recompile: make regen-importlib && make
* Run test_concurrency() just once: ./python -m test -m test_concurrency -v test_import
* You get the bug
By the way, importlib_sleep.patch changes test_concurrency() to run the test exactly once (it changes to loop to a single iteration, instead of 10).
Is it normal that a sleep makes import failing? Imports are not supposed to be atomic?
--
I tried to similar hack before the commit b4baacee1adc06edbe30ac7574d17a8cd168e2e0:
* add a sleep in importlib
* copy test_concurrency() from master
* Run test_concurrency()
* The test pass
While I'm not sure of my hack, it *seems* like sys.modules was somehow atomic before this commit? |
|
Date |
User |
Action |
Args |
2017-07-10 13:59:46 | vstinner | set | recipients:
+ vstinner, brett.cannon, paul.moore, ncoghlan, tim.golden, eric.snow, zach.ware, serhiy.storchaka, steve.dower |
2017-07-10 13:59:46 | vstinner | set | messageid: <1499695186.19.0.0575292245907.issue30891@psf.upfronthosting.co.za> |
2017-07-10 13:59:46 | vstinner | link | issue30891 messages |
2017-07-10 13:59:46 | vstinner | create | |
|