This issue tracker has been migrated to GitHub, and is currently read-only.
For more information, see the GitHub FAQs in the Python's Developer Guide.

classification
Title: test_importlib: test_locks failure
Type: behavior Stage: resolved
Components: Tests Versions: Python 3.3, Python 3.4
process
Status: closed Resolution: fixed
Dependencies: Superseder:
Assigned To: Nosy List: brett.cannon, christian.heimes, eric.snow, georg.brandl, ncoghlan, pitrou, python-dev, skrah, vstinner
Priority: normal Keywords: patch

Created on 2012-08-27 22:35 by skrah, last changed 2022-04-11 14:57 by admin. This issue is now closed.

Files
File name Uploaded Description Edit
relax_deadlock_test.patch pitrou, 2012-08-28 00:05 review
Messages (5)
msg169232 - (view) Author: Stefan Krah (skrah) * (Python committer) Date: 2012-08-27 22:35
On the obstinate i7 machine (see #15781), test_importlib fails
sporadically (after 10 repetitions or so).

$ ./python -m test -uall -F test_importlib                                                               
[  1] test_importlib                                                                                     
test test_importlib failed -- Traceback (most recent call last):                                         
  File "/home/stefan/hg/cpython/Lib/test/test_importlib/test_locks.py", line 80, in test_deadlock        
    self.assertEqual(results.count((True, False)), 1)                                                    
AssertionError: 2 != 1                                                                                   
                                                                                                         
1 test failed:                                                                                           
    test_importlib
msg169238 - (view) Author: Stefan Krah (skrah) * (Python committer) Date: 2012-08-27 22:47
Also reproduced on Ubuntu Lucid/Core 2 Duo with a very low switch
interval and -j4. It takes more repetitions than on the i7:

$ ./python -m test -uall -F -j4  test_importlib 
[...]
[224] test_importlib
[225] test_importlib
[226/1] test_importlib
test test_importlib failed -- Traceback (most recent call last):
  File "/home/stefan/pydev/cpython/Lib/test/test_importlib/test_locks.py", line 80, in test_deadlock
    self.assertEqual(results.count((True, False)), 1)
AssertionError: 2 != 1
msg169239 - (view) Author: Antoine Pitrou (pitrou) * (Python committer) Date: 2012-08-28 00:05
After a bit of investigation, it turns out the test is probably too strict.
The deadlock avoidance logic in importlib._bootstrap is indeed conservative: if you take a look at _ModuleLock.acquire(), _blocking_on[tid] is set even while other threads may run, because the internal lock is not always taken. Therefore, if there's a potential deadlock between several threads, there are situations where DeadlockError will be raised in two threads, not only one.

(IMHO it would complicate the code too much to make deadlock avoidance less conservative, without bringing anything in terms of additional functionality)

See attached patch.
msg169257 - (view) Author: Stefan Krah (skrah) * (Python committer) Date: 2012-08-28 09:00
The patch works fine here (as expected).
msg169290 - (view) Author: Roundup Robot (python-dev) (Python triager) Date: 2012-08-28 18:13
New changeset 454dceb5fd56 by Antoine Pitrou in branch 'default':
Issue #15794: Relax a test case due to the deadlock detection's conservativeness.
http://hg.python.org/cpython/rev/454dceb5fd56
History
Date User Action Args
2022-04-11 14:57:35adminsetgithub: 59998
2012-08-28 18:14:47pitrousetstatus: open -> closed
resolution: fixed
stage: resolved
2012-08-28 18:13:55python-devsetnosy: + python-dev
messages: + msg169290
2012-08-28 09:00:49skrahsetmessages: + msg169257
2012-08-28 00:05:20pitrousetfiles: + relax_deadlock_test.patch
keywords: + patch
messages: + msg169239
2012-08-27 22:47:18skrahsetmessages: + msg169238
2012-08-27 22:35:01skrahcreate