classification
Title: '_DummyThread' object has no attribute '_Thread__block'
Type: behavior Stage: committed/rejected
Components: Versions: Python 3.3, Python 3.2, Python 2.7
process
Status: closed Resolution: fixed
Dependencies: Superseder:
Assigned To: Nosy List: Dustin.Kirkland, Stephen.White, cooyeah, neologix, pitrou, python-dev, sbt
Priority: normal Keywords: patch

Created on 2012-03-14 23:48 by Dustin.Kirkland, last changed 2012-05-09 14:55 by Stephen.White. This issue is now closed.

Files
File name Uploaded Description Edit
thread.py.patch Dustin.Kirkland, 2012-03-14 23:48
dummythreadafterfork.patch pitrou, 2012-04-19 13:40
dummythreadafterfork2.patch pitrou, 2012-04-19 21:21
bad-thread.py Stephen.White, 2012-05-09 14:55 Minimal failing example
Messages (18)
msg155818 - (view) Author: Dustin Kirkland (Dustin.Kirkland) Date: 2012-03-14 23:48
My Apache2 logs are filled with the following error kicked out by my python wsgi script:

[Wed Mar 14 18:16:38 2012] [error] Exception AttributeError: AttributeError("'_DummyThread' object has no attribute '_Thread__block'",) in <module 'threading' from '/usr/lib/python2.7/threading.pyc'> ignored

I was able to silence these with a simple conditional in /usr/lib/python2.7/threading.pyc:

-        del self._Thread__block
+        if hasattr(self, '_Thread__block'):
+            del self._Thread__block

Full patch attached.
msg155882 - (view) Author: Antoine Pitrou (pitrou) * (Python committer) Date: 2012-03-15 12:45
I don't understand how that can happen, since _Thread__block is initialized in Thread.__init__, which is called by _DummyThread.__init__. Did you somehow monkeypatch the threading module?
msg155884 - (view) Author: Dustin Kirkland (Dustin.Kirkland) Date: 2012-03-15 14:06
/usr/lib/python2.7/threading.pyc is stock from my distribution, Ubuntu 12.04 LTS, which has python-2.7.3~rc1-1ubuntu2.

I did manually apply the patch I've attached here to /usr/lib/python2.7/threading.py, and it *seemed* to silence those errors.

However, on checking my logs again this morning, I have a few hundred more of the same error, so that was not effective.

First time contributor here...what would be the proper way to test this?

Thanks for your patience.
msg155886 - (view) Author: Antoine Pitrou (pitrou) * (Python committer) Date: 2012-03-15 14:18
> However, on checking my logs again this morning, I have a few hundred
> more of the same error, so that was not effective.

Did you restart your server?

In your patch, you could use the traceback module (e.g. http://docs.python.org/dev/library/traceback.html#traceback.print_stack ) to figure out in which situation the error occurs.
msg155887 - (view) Author: Richard Oudkerk (sbt) * (Python committer) Date: 2012-03-15 14:25
_DummyThread.__init__() explicitly deletes self._Thread__block:

    def __init__(self):
        Thread.__init__(self, name=_newname("Dummy-%d"))

        # Thread.__block consumes an OS-level locking primitive, which
        # can never be used by a _DummyThread.  Since a _DummyThread
        # instance is immortal, that's bad, so release this resource.
        del self._Thread__block
        ^^^^^^^^^^^^^^^^^^^^^^^
msg155889 - (view) Author: Richard Oudkerk (sbt) * (Python committer) Date: 2012-03-15 14:29
Ignore my last message...
msg156503 - (view) Author: Dustin Kirkland (Dustin.Kirkland) Date: 2012-03-21 17:55
Okay, update...

I did rebuild all of Python from source (actually, I applied it to the Ubuntu python2.7 package, rebuilt that locally, and then upgraded to the new python2.7 deb's.  I could see my change was applied /usr/lib/python2.7/threading.py, and I can safely assume that it landed in the pyc as well.

I restarted Apache2 to reload my wsgi script.  But unfortunately, I get the same error in /var/log/apache/error.log:

[Wed Mar 21 12:52:36 2012] [error] Exception AttributeError: AttributeError("'_DummyThread' object has no attribute '_Thread__block'"
,) in <module 'threading' from '/usr/lib/python2.7/threading.pyc'> ignored                                                           

So please consider my patch as wrong/bad/insufficient.

Issue is still open, though.  Thanks.
msg158697 - (view) Author: (cooyeah) Date: 2012-04-19 04:35
I saw the similar problem on Ubuntu 12.04 with django development server with mediageneartor. As Dustin suggested, I don't think this is related to the "del _Thread__block" statement.

I hacked the __getattribute__ of DummyThread class

    def __getattribute__(self, name):
        if name == '_Thread__block':
            import traceback
            traceback.print_stack()
            raise AttributeError
        return Thread.__getattribute__(self, name)

And I got the following stacktrace:

  File "/tmp/ve/local/lib/python2.7/site-packages/mediagenerator/filters/coffeescript.py", line 54, in _compile
    shell=shell, universal_newlines=True)
  File "/usr/lib/python2.7/subprocess.py", line 679, in __init__
    errread, errwrite)
  File "/usr/lib/python2.7/subprocess.py", line 1143, in _execute_child
    self.pid = os.fork()
  File "/usr/lib/python2.7/threading.py", line 907, in _after_fork
    thread._Thread__stop()
  File "/usr/lib/python2.7/threading.py", line 608, in __stop
    self.__block.acquire()
  File "/usr/lib/python2.7/threading.py", line 827, in __getattribute__
    traceback.print_stack()
msg158725 - (view) Author: Antoine Pitrou (pitrou) * (Python committer) Date: 2012-04-19 13:19
Ok, could you try applying the following patch to threading.py?


diff --git a/Lib/threading.py b/Lib/threading.py
--- a/Lib/threading.py
+++ b/Lib/threading.py
@@ -887,7 +887,7 @@ def _after_fork():
                 ident = _get_ident()
                 thread._Thread__ident = ident
                 new_active[ident] = thread
-            else:
+            elif not isinstance(thread, _DummyThread):
                 # All the others are already stopped.
                 thread._Thread__stop()
msg158726 - (view) Author: Antoine Pitrou (pitrou) * (Python committer) Date: 2012-04-19 13:36
Here is a complete patch + tests for 2.7.
msg158754 - (view) Author: Charles-François Natali (neologix) * (Python committer) Date: 2012-04-19 21:11
> Here is a complete patch + tests for 2.7.

I like the test.

However there's something I find strange with the patch:
"""
diff --git a/Lib/threading.py b/Lib/threading.py
--- a/Lib/threading.py
+++ b/Lib/threading.py
@@ -887,7 +887,7 @@ def _after_fork():
                 ident = _get_ident()
                 thread._Thread__ident = ident
                 new_active[ident] = thread
-            else:
+            elif not isinstance(thread, _DummyThread):
                 # All the others are already stopped.
                 thread._Thread__stop()
"""

Is it really the caller's job to check that the thread is not a dummy thread?
IMO it should be _DummyThread's stop() method that does the right thing, either by overriding Thread's stop() method in _DummyThread or by puting the check inside Thread.stop(), like what's done inside thread._reset_internal_locks():
"""
         if hasattr(self, '_Thread__block'):  # DummyThread deletes self.__block
             self.__block.__init__()
             self.__started._reset_internal_locks()
"""
msg158756 - (view) Author: Antoine Pitrou (pitrou) * (Python committer) Date: 2012-04-19 21:17
Le jeudi 19 avril 2012 à 21:11 +0000, Charles-François Natali a écrit :
> IMO it should be _DummyThread's stop() method that does the right
> thing, either by overriding Thread's stop() method in _DummyThread or
> by puting the check inside Thread.stop(), like what's done inside
> thread._reset_internal_locks():

I don't think _DummyThread can override __stop(), because of the name
mangling of __private methods. However, the hasattr() approach would
probably work.
msg158757 - (view) Author: Antoine Pitrou (pitrou) * (Python committer) Date: 2012-04-19 21:21
New patch with the hasattr() approach.
msg158761 - (view) Author: Charles-François Natali (neologix) * (Python committer) Date: 2012-04-19 21:45
> New patch with the hasattr() approach.

LGTM.
msg158762 - (view) Author: Roundup Robot (python-dev) Date: 2012-04-19 22:06
New changeset ab9d6c4907e7 by Antoine Pitrou in branch '2.7':
Issue #14308: Fix an exception when a "dummy" thread is in the threading module's active list after a fork().
http://hg.python.org/cpython/rev/ab9d6c4907e7

New changeset 41c64c700e1e by Antoine Pitrou in branch '3.2':
Issue #14308: Fix an exception when a "dummy" thread is in the threading module's active list after a fork().
http://hg.python.org/cpython/rev/41c64c700e1e

New changeset e3ea462cb181 by Antoine Pitrou in branch 'default':
Issue #14308: Fix an exception when a dummy thread is in the threading module's active list after a fork().
http://hg.python.org/cpython/rev/e3ea462cb181
msg158763 - (view) Author: Richard Oudkerk (sbt) * (Python committer) Date: 2012-04-19 22:08
> I don't think _DummyThread can override __stop(), because of the name
> mangling of __private methods. However, the hasattr() approach would
> probably work.

Wouldn't a _DummyThread._Thread__stop() method override Thread.__stop()?  Like

>>> class A(object):
...     def foo(self):
...         self.__bar()
...     def __bar(self):
...         print "original"
...
>>> class B(A):
...     def _A__bar(self):
...         print "overridden"
...
>>> B().foo()
overridden
msg158765 - (view) Author: Antoine Pitrou (pitrou) * (Python committer) Date: 2012-04-19 22:10
> Wouldn't a _DummyThread._Thread__stop() method override Thread.__stop()?

Probably, but that would be quite ugly IMHO.
I've now committed the patch as-is in 2.7. In 3.2 it turned out easier: __stop is now spelt _stop, so can be overriden without any hacks.
msg160297 - (view) Author: Stephen White (Stephen.White) Date: 2012-05-09 14:55
Glad this is fixed.  Attached is a Python 2.7 file that demonstrates the problem in a pretty minimal way in case it is of any use to anyone.
History
Date User Action Args
2012-05-09 14:55:36Stephen.Whitesetfiles: + bad-thread.py
nosy: + Stephen.White
messages: + msg160297

2012-04-19 22:10:59pitrousetstatus: open -> closed
resolution: fixed
messages: + msg158765

stage: patch review -> committed/rejected
2012-04-19 22:08:04sbtsetmessages: + msg158763
2012-04-19 22:06:43python-devsetnosy: + python-dev
messages: + msg158762
2012-04-19 21:45:15neologixsetmessages: + msg158761
2012-04-19 21:21:58pitrousetfiles: + dummythreadafterfork2.patch

messages: + msg158757
2012-04-19 21:17:02pitrousetmessages: + msg158756
2012-04-19 21:11:13neologixsetnosy: + neologix
messages: + msg158754
2012-04-19 13:40:57pitrousetfiles: + dummythreadafterfork.patch
2012-04-19 13:40:50pitrousetfiles: - dummythreadafterfork.patch
2012-04-19 13:36:05pitrousetfiles: + dummythreadafterfork.patch

messages: + msg158726
2012-04-19 13:19:37pitrousetmessages: + msg158725
2012-04-19 04:35:55cooyeahsetnosy: + cooyeah
messages: + msg158697
2012-03-21 17:55:47Dustin.Kirklandsetmessages: + msg156503
2012-03-15 14:29:00sbtsetmessages: + msg155889
2012-03-15 14:25:10sbtsetnosy: + sbt
messages: + msg155887
2012-03-15 14:18:59pitrousetmessages: + msg155886
2012-03-15 14:06:39Dustin.Kirklandsetmessages: + msg155884
2012-03-15 12:45:01pitrousetnosy: + pitrou
messages: + msg155882
2012-03-15 11:30:31pitrousetstage: test needed -> patch review
2012-03-14 23:53:19r.david.murraysettitle: Exception AttributeError: AttributeError("'_DummyThread' object has no attribute '_Thread__block'",) in <module 'threading' from '/usr/lib/python2.7/threading.pyc'> ignored -> '_DummyThread' object has no attribute '_Thread__block'
stage: test needed
type: crash -> behavior
versions: + Python 3.2, Python 3.3
2012-03-14 23:48:20Dustin.Kirklandcreate