This issue tracker has been migrated to GitHub, and is currently read-only.
For more information, see the GitHub FAQs in the Python's Developer Guide.

Title: Import deadlock detection causes deadlock
Type: behavior Stage: patch review
Components: Library (Lib) Versions: Python 3.9, Python 3.8, Python 3.7, Python 3.6
Status: open Resolution:
Dependencies: Superseder:
Assigned To: Nosy List: Ronan.Lamy, brett.cannon, eric.snow, miss-islington, pconnell, pitrou, vzhestkov
Priority: normal Keywords: patch

Created on 2019-09-10 13:26 by Ronan.Lamy, last changed 2022-04-11 14:59 by admin.

Pull Requests
URL Status Linked Edit
PR 17518 merged arigo, 2019-12-09 10:27
Messages (3)
msg351647 - (view) Author: Ronan Lamy (Ronan.Lamy) * Date: 2019-09-10 13:26
There seems to be a race condition in importlib._bootstrap._ModuleLock that can cause a deadlock. The sequence of operations is as follows:

* Thread 1 calls lock.acquire()
* Thread 1 sets itself as lock.owner and begins importing the module
* Thread 2 calls lock.acquire() and waits for lock.lock
* Thread 2 gets lock.lock
* Thread 1 calls lock.acquire() again, due to a nested import
* Thread 1 sets itself as blocking on lock: _blocking_on[tid1] = lock
* Thread 2 enters lock.has_deadlock()
* Thread 2 busy-waits forever in has_deadlock() because lock.owner == tid1 and _blocking_on[tid1] == lock
* Thread 1 waits forever for lock.lock since thread 2 owns it

The issue was found in pypy3 but it also affects all the recent CPython versions I tried.
I can reliably reproduce the issue by adding an artificial delay to _ModuleLock.has_deadlock(), e.g. with this patch:

diff --git a/Lib/test/test_import/ b/Lib/test/test_import/
index f167c84..7f7188e 100644
--- a/Lib/test/test_import/
+++ b/Lib/test/test_import/
@@ -435,10 +435,15 @@ class ImportTests(unittest.TestCase):
     def test_concurrency(self):
+        def delay_has_deadlock(frame, event, arg):
+            if event == 'call' and frame.f_code.co_name == 'has_deadlock':
+                time.sleep(0.2)
         sys.path.insert(0, os.path.join(os.path.dirname(__file__), 'data'))
             exc = None
             def run():
+                sys.settrace(delay_has_deadlock)
                     import package
msg363232 - (view) Author: miss-islington (miss-islington) Date: 2020-03-03 01:37
New changeset 6daa37fd42c5d5300172728e8b4de74fe0b319fc by Armin Rigo in branch 'master':
bpo-38091: Import deadlock detection causes deadlock (GH-17518)
msg397884 - (view) Author: Victor Zhestkov (vzhestkov) Date: 2021-07-20 16:00
I ported the fix from for 3.6 and 3.8 shipped with SLE 15SP2 and openSUSE Tumbleweed, but it seems that this fix doesn't help.
I have a deadlocks on running `salt-api` process managing `salt-ssh` systems with high workload. The service can get the deadlock in first 5 minutes or after 3-60 minutes of running the service with the same workload with almost equal chances.

Here is the part of py-bt I see each time:

(gdb) py-bt
Traceback (most recent call first):
  File "<frozen importlib._bootstrap>", line 107, in acquire
  File "<frozen importlib._bootstrap>", line 158, in __enter__
  File "<frozen importlib._bootstrap>", line 595, in _exec
  File "<frozen importlib._bootstrap>", line 271, in _load_module_shim
  File "<frozen importlib._bootstrap_external>", line 852, in load_module
  File "<frozen importlib._bootstrap_external>", line 1027, in load_module
  File "<frozen importlib._bootstrap_external>", line 1034, in _check_name_wrapper
  File "/usr/lib/python3.8/site-packages/salt/", line 4779, in _load_module
  File "/usr/lib/python3.8/site-packages/salt/", line 1926, in _inner_load
    if self._load_module(name) and key in self._dict:
  File "/usr/lib/python3.8/site-packages/salt/", line 2193, in _load
  File "/usr/lib/python3.8/site-packages/salt/utils/", line 99, in __getitem__
    if self._load(key):
  File "/usr/lib/python3.8/site-packages/salt/", line 1283, in __getitem__
    func = super().__getitem__(item)
  File "/usr/lib/python3.8/site-packages/salt/", line 1139, in __getitem__
    return self._dict[key + self.suffix]
  File "/usr/lib/python3.8/site-packages/salt/", line 495, in check_render_pipe_str
  File "/usr/lib/python3.8/site-packages/salt/", line 1428, in render
  File "/usr/lib/python3.8/site-packages/salt/pillar/", line 781, in __init__
Date User Action Args
2022-04-11 14:59:20adminsetgithub: 82272
2021-07-20 16:00:14vzhestkovsetnosy: + vzhestkov
messages: + msg397884
2020-03-03 01:37:31miss-islingtonsetnosy: + miss-islington
messages: + msg363232
2019-12-09 10:27:17arigosetkeywords: + patch
stage: patch review
pull_requests: + pull_request16996
2019-10-28 20:47:46pconnellsetnosy: + pconnell
2019-09-11 14:04:05eric.snowsetnosy: + brett.cannon, eric.snow
2019-09-11 13:10:14eric.snowsetnosy: + pitrou
2019-09-10 13:26:40Ronan.Lamycreate