This issue tracker has been migrated to GitHub, and is currently read-only.
For more information, see the GitHub FAQs in the Python's Developer Guide.

classification
Title: importlib race condition
Type: Stage: resolved
Components: Interpreter Core Versions: Python 3.7
process
Status: closed Resolution: not a bug
Dependencies: Superseder:
Assigned To: Nosy List: brett.cannon, eric.snow, ncoghlan, serhiy.storchaka, vstinner
Priority: normal Keywords:

Created on 2017-08-22 12:26 by vstinner, last changed 2022-04-11 14:58 by admin. This issue is now closed.

Messages (2)
msg300686 - (view) Author: STINNER Victor (vstinner) * (Python committer) Date: 2017-08-22 12:26
While stressing the system to get system load of at least 5 (I have 4 logical CPUs), I got an import error whereas the module exists.

See recent changes to fix race conditions in importlib:

* bpo-30891: commit e72b1359f81d1dd42bd8a5c5cc2b3928b74f8023
* bpo-30891: commit 4f9a446f3fb42f800e73cd9414dd1eccb3ca4fa7
* bpo-31070: commit 9b0d1d647e3d2ec9d299e5c9f49b02fbbb810a5a
* bpo-30814: commit b4baacee1adc06edbe30ac7574d17a8cd168e2e0
* ...

I used my system_load.py script to stress my Linux:
https://github.com/haypo/misc/blob/master/bin/system_load.py


haypo@selma$ ./python -m test -r  --randseed=7323369 -R 3:3 -j2 -m test.test_robotparser.PasswordProtectedSiteTestCase.testPasswordProtectedSite test_robotparser  test_robotparser  test_robotparser  test_robotparsertest_robotparser  test_robotparser  test_robotparser  test_robotparser
Using random seed 7323369
Run tests in parallel using 2 child processes
0:00:01 load avg: 9.05 [1/7] test_robotparser passed
beginning 6 repetitions
123456
......
0:00:01 load avg: 9.05 [2/7] test_robotparser passed
beginning 6 repetitions
123456
......
0:00:02 load avg: 10.33 [3/7] test_robotparser passed
beginning 6 repetitions
123456
......
0:00:02 load avg: 10.33 [4/7] test_robotparser passed
beginning 6 repetitions
123456
......
0:00:03 load avg: 10.33 [5/7/1] test_robotparsertest_robotparser failed
test test_robotparsertest_robotparser crashed -- Traceback (most recent call last):
  File "/home/haypo/prog/python/master/Lib/test/libregrtest/runtest.py", line 163, in runtest_inner
    the_module = importlib.import_module(abstest)
  File "/home/haypo/prog/python/master/Lib/importlib/__init__.py", line 127, in import_module
    return _bootstrap._gcd_import(name[level:], package, level)
  File "<frozen importlib._bootstrap>", line 994, in _gcd_import
  File "<frozen importlib._bootstrap>", line 971, in _find_and_load
  File "<frozen importlib._bootstrap>", line 953, in _find_and_load_unlocked
ModuleNotFoundError: No module named 'test.test_robotparsertest_robotparser'
0:00:03 load avg: 10.33 [6/7/1] test_robotparser passed
beginning 6 repetitions
123456
......
0:00:04 load avg: 10.33 [7/7/1] test_robotparser passed
beginning 6 repetitions
123456
......
6 tests OK.

1 test failed:
    test_robotparsertest_robotparser

Total duration: 4 sec
Tests result: FAILURE
msg300688 - (view) Author: STINNER Victor (vstinner) * (Python committer) Date: 2017-08-22 12:41
Oops, please ignore my issue. It's a typo in my command line, missing space ;-)
History
Date User Action Args
2022-04-11 14:58:51adminsetgithub: 75440
2017-08-22 12:41:18vstinnersetstatus: open -> closed
resolution: not a bug
messages: + msg300688

stage: resolved
2017-08-22 12:29:57vstinnersetcomponents: + Interpreter Core
versions: + Python 3.7
2017-08-22 12:26:54vstinnercreate