This issue tracker has been migrated to GitHub, and is currently read-only.
For more information, see the GitHub FAQs in the Python's Developer Guide.

classification
Title: Python2.x sre and threaded import lockin
Type: Stage:
Components: Interpreter Core Versions:
process
Status: closed Resolution: wont fix
Dependencies: Superseder:
Assigned To: tim.peters Nosy List: effbot, jpkelly, tim.peters
Priority: low Keywords:

Created on 2001-08-24 03:40 by jpkelly, last changed 2022-04-10 16:04 by admin. This issue is now closed.

Messages (4)
msg6157 - (view) Author: Joe Kelly (jpkelly) Date: 2001-08-24 03:40
There appears to be a bug in the way Python2.x does 
import locking on
sre.  This is illustrated by the following code 
fragments.

$ cat retest
#!/usr/local/bin/python
__import__("ret")

$ cat ret.py
#!/usr/local/bin/python

import time, thread, string
#import time, thread
import re
#import pre
#re=pre

def mythread():
        time.sleep(2)
        url = "hello there"
        url = re.sub(" ", '', url)
        print "hello from threadland", url

thread.start_new_thread(mythread, ())

url = "hello main there"
url = re.sub(" ", '', url)
print "main", url
time.sleep(5)
print "goodbye", url


If you run retest as shown above you get:
$ retest
main hellomainthere
goodbye hellomainthere

It never runs the code in the thread.

If you run python on ret.pyc directly you get the 
correct/expected
result:
$ python ret.pyc
main hellomainthere
hello from threadland hellothere
goodbye hellomainthere

If you replace sre with pre by commenting out line 5 
and uncommenting
out line 6 and 7 it works correctly/as expected:
$ retest
main hellomainthere
hello from threadland hellothere
goodbye hellomainthere

I've also noticed some strange interaction with other 
imports, i.e. if
you go back to the original ret.py and comment out 
line 3 and
uncomment line 4 (just don't import the string module) 
you get the
following:
$ retest
main hellomainthere
goodbye hellomainthere
hello from threadland hellothere

That is the thread gets executed at the end or 
sometimes not at all:
$ retest
main hellomainthere
goodbye hellomainthere

Again, calling ret directly seems to work.
$ python ret.pyc
main hellomainthere
hello from threadland hellothere
goodbye hellomainthere

This has been tested on Solaris 8 running on Intel:
$ uname -a
SunOS machine 5.8 Generic_108529-07 i86pc i386 i86pc

It also exhibits the same behavior under Linux:
$ uname -a
Linux machine 2.4.2-2smp #1 SMP Sun Apr 8 20:21:34 EDT 
2001 i686 unknown

The above output was generated by python 2.0 compiled 
with threading
on:
$ python
Python 2.0 (#1, May 13 2001, 00:55:08)
[GCC 2.95.2 19991024 (release)] on sunos5
Type "copyright", "credits" or "license" for more 
information.
>>>

It exhibits the same behavior under python 2.1.1:
$ python
Python 2.1.1 (#22, Aug 20 2001, 17:12:35)
[GCC 2.95.2 19991024 (release)] on sunos5
Type "copyright", "credits" or "license" for more 
information.

msg6158 - (view) Author: Tim Peters (tim.peters) * (Python committer) Date: 2001-08-24 04:20
Logged In: YES 
user_id=31435

Assigned to Fredrik, but as I said on c.l.py, never write a 
module that spawns threads as a side effect of merely being 
imported.  Everything you're chasing is an accident, and 
while /F may or may not bother worming around this one, 
you're going to get in trouble over and over again.  I've 
reduced the priority accordingly.

/F, the immediate cause for this one is the

        import sre_parse

in the body of sre_compile.compile(), which is reached in 
the spawned thread via calling re.sub().  Since the 
module "ret" is itself imported, the main thread doing the 
import of "ret" holds the import lock for the duration.  
This causes the spawned thread (created by merely 
importing "ret", and there's no proper synchronization in 
the test case either, so accidents compound) to block on 
any import attempt.  In particular, the spawned thread 
blocks on the line shown above.  Worm around that, and it 
blocks somewhere else later; etc.  In *general* it's 
hopeless.
msg6159 - (view) Author: Fredrik Lundh (effbot) * (Python committer) Date: 2001-09-18 18:52
Logged In: YES 
user_id=38376

Reduced priority (to match "hopeless" status)
msg6160 - (view) Author: Tim Peters (tim.peters) * (Python committer) Date: 2001-09-18 20:03
Logged In: YES 
user_id=31435

Reassigned to me and closed as WontFix.  I confirmed with 
Guido that we have no bright ideas pending for worming 
around cases like this, and the import lock remains 
essential to prevent worse problems.  "Never write a 
module that spawns threads as a side effect of merely being 
imported" remains the only reliable workaround (e.g., spawn 
the threads from a module function called after the module 
import completes).

BTW, 2.2 will contain (& CVS Python already does) a new 
imp.lock_held() function, so you can determine whether the 
import lock is currently held.
History
Date User Action Args
2022-04-10 16:04:22adminsetgithub: 35037
2001-08-24 03:40:02jpkellycreate