Back in time, the function PyImport_ImportModuleNoBlock() was introduced and used in modules such as the time module to supposedly avoid deadlocks when using threads. It may well have solved that problem, but only served to cause other problems.

To illustrate the problem consider the test code:


import imp
import thread
import time

def run1():
   print 'acquire'
   imp.acquire_lock()
   time.sleep(5)
   imp.release_lock()
   print 'release'

thread.start_new_thread(run1, ())

time.sleep(2)

print 'strptime'
time.strptime("", "")
print 'exit'


The output of running this is


grumpy:~ grahamd$ python noblock.py 
acquire
strptime
Traceback (most recent call last):
  File "noblock.py", line 17, in <module>
    time.strptime("", "")
ImportError: Failed to import _strptime because the import lockis held by another thread.


It is bit silly that code executing in one thread could fail because at the time that it tries to call time.strptime() a different thread has the global import lock.

This problem may not arise in applications which preload all modules, but it will where importing of modules is deferred until later within execution of a thread and where there may be concurrent threads running doing work that requires modules imported by that new C function.

Based on old discussion at:

http://groups.google.com/group/comp.lang.python/browse_frm/thread/dad73ac47b81a744

my expectation is that this issue will be rejected as not being a problem with any remedy being pushed to the application developer. Personally I don't agree with that and believe that the real solution is to come up with an alternate fix for the original deadlock that doesn't introduce this new detrimental behaviour. This may entail structural changes to modules such as the time module to avoid issue.

Unfortunately since the PyImport_ImportModuleNoBlock() function has been introduced, it is starting to be sprinkled like fairy dust across modules in the standard library and in third party modules. This is only going to set up future problems in multithreaded applications, especially where third party module developers don't appreciate what problems they are potentially introducing by using this function.

Anyway, not optimistic from what I have seen that this will be changed, so view this as a protest against this behaviour. :-)

FWIW, issue in mod_wsgi issue tracker about this is:

http://code.google.com/p/modwsgi/issues/detail?id=177

I have known about this issue since early last year though.