classification
Title: test_threading_local sometimes hangs when run with -R
Type: crash Stage: resolved
Components: Library (Lib), Tests Versions: Python 3.2, Python 3.1, Python 2.7, Python 2.6
process
Status: closed Resolution: fixed
Dependencies: Superseder:
Assigned To: Nosy List: gps, gregory.p.smith, pitrou
Priority: normal Keywords: patch

Created on 2009-11-04 20:04 by pitrou, last changed 2009-11-05 14:42 by pitrou. This issue is now closed.

Files
File name Uploaded Description Edit
threading_local.patch pitrou, 2009-11-04 20:23
threading_local2.patch pitrou, 2009-11-04 23:24
Messages (6)
msg94900 - (view) Author: Antoine Pitrou (pitrou) * (Python committer) Date: 2009-11-04 20:04
If I run something like:
  ./python -m test.regrtest -R3:2: test_threading_local

Python sometimes hangs and I have to "kill -9" it.

Running through gdb shows it gets stuck at the following point:

/home/antoine/cpython/debug/Lib/threading.py (239): wait
/home/antoine/cpython/debug/Lib/threading.py (638): join
/home/antoine/cpython/debug/Lib/test/test_threading_local.py (68):
test_derived
[snip]

That is, it hangs waiting for a thread to join().
msg94901 - (view) Author: Antoine Pitrou (pitrou) * (Python committer) Date: 2009-11-04 20:23
It turns out that the __del__ method in _threading_local.py tries to
call threading.enumerate() which itself takes the _active_limbo_lock.
The problem is that __del__ can be called at any point in time (because
of the GC), including at a point where the same thread has already taken
the lock. The obvious fix is to bypass enumerate().

(an alternate fix would be to use an RLock for _active_limbo_lock, but
it could have unforeseen consequences, such as performance ones)
msg94908 - (view) Author: Gregory P. Smith (gregory.p.smith) * (Python committer) Date: 2009-11-04 23:19
How about defining this in threading.py:

def _enumerate():
    """Internal use only: enumerate() without the lock."""
    return _active.values() + _limbo.values()

And calling it from _threading_local instead of accessing _active and
_limbo directly.
msg94909 - (view) Author: Antoine Pitrou (pitrou) * (Python committer) Date: 2009-11-04 23:24
Good point. Here is a new patch.
msg94922 - (view) Author: Gregory P. Smith (gregory.p.smith) * (Python committer) Date: 2009-11-05 06:32
Looks good to me.

typed poorly on a tiny Android phone.

On Nov 4, 2009 3:24 PM, "Antoine Pitrou" <report@bugs.python.org> wrote:

Antoine Pitrou <pitrou@free.fr> added the comment:

Good point. Here is a new patch.

----------
Added file: http://bugs.python.org/file15264/threading_local2.patch

_______________________________________ Python tracker <
report@bugs.python.org> <http://bugs.python...
msg94931 - (view) Author: Antoine Pitrou (pitrou) * (Python committer) Date: 2009-11-05 14:42
Committed, thanks.
History
Date User Action Args
2009-11-05 14:42:14pitrousetstatus: open -> closed
resolution: fixed
stage: patch review -> resolved
2009-11-05 14:42:04pitrousetmessages: + msg94931
2009-11-05 13:38:04pitrousetfiles: - unnamed
2009-11-05 06:32:21gregory.p.smithsetfiles: + unnamed

messages: + msg94922
2009-11-04 23:24:39pitrousetfiles: + threading_local2.patch

messages: + msg94909
2009-11-04 23:19:01gregory.p.smithsetnosy: + gregory.p.smith
messages: + msg94908
2009-11-04 20:24:10pitrousetversions: + Python 2.6, Python 3.1, Python 3.2
2009-11-04 20:23:59pitrousetstage: patch review
2009-11-04 20:23:10pitrousetfiles: + threading_local.patch
keywords: + patch
messages: + msg94901
2009-11-04 20:04:15pitroucreate