classification
Title: multiprocessing.dummy craches when self._parent._children does not exist
Type: behavior Stage: resolved
Components: Library (Lib) Versions: Python 3.2, Python 3.3, Python 2.7
process
Status: closed Resolution: fixed
Dependencies: Superseder:
Assigned To: Nosy List: Andres.Riancho, Itay.Brandes, python-dev, sbt, terry.reedy
Priority: normal Keywords:

Created on 2012-05-22 08:51 by Itay.Brandes, last changed 2013-10-19 00:33 by terry.reedy. This issue is now closed.

Messages (7)
msg161339 - (view) Author: Itay Brandes (Itay.Brandes) Date: 2012-05-22 08:51
multiprocessing.dummy crashes when attempting to create a ThreadPool from a Thread.

The following code will crush on 2.7.3:

    import multiprocessing.dummy
    import threading
     
    class Worker(threading.Thread):
        def __init__(self):
            threading.Thread.__init__(self)
     
        def run(self):
            poll = multiprocessing.dummy.Pool(5)
            print str(poll)
     
    w = Worker()
    w.start()

will crush with the following traceback:

    poll = multiprocessing.dummy.Pool(5)
  File "C:\Python27\lib\multiprocessing\dummy\__init__.py", line 150, in Pool
    return ThreadPool(processes, initializer, initargs)
  File "C:\Python27\lib\multiprocessing\pool.py", line 685, in __init__
    Pool.__init__(self, processes, initializer, initargs)
  File "C:\Python27\lib\multiprocessing\pool.py", line 136, in __init__
    self._repopulate_pool()
  File "C:\Python27\lib\multiprocessing\pool.py", line 199, in _repopulate_pool
    w.start()
  File "C:\Python27\lib\multiprocessing\dummy\__init__.py", line 73, in start
    self._parent._children[self] = None
AttributeError: 'Worker' object has no attribute '_children'

If you have access to the thread itself, you can set the _children attribute youself (w._children = weakref.WeakKeyDictionary()), but it is not possible with threads different from threading.thread (Such as PyQt4's QThread thread, which is essential for GUI programming).

The fix that I found is to edit the Python27\Lib\multiprocessing\dummy\__init__.py file. The crashing code is line number 73.
This line should be nested in an if block:

        if hasattr(self._parent, '_children'):
            self._parent._children[self] = None

That way the code is fixed.
Thanks!
msg161576 - (view) Author: Roundup Robot (python-dev) (Python triager) Date: 2012-05-25 12:06
New changeset 1f5d2642929a by Richard Oudkerk in branch '2.7':
Issue #14881: Allow normal non-main thread to spawn a dummy process
http://hg.python.org/cpython/rev/1f5d2642929a

New changeset 0528ec18e230 by Richard Oudkerk in branch '3.2':
Issue #14881: Allow normal non-main thread to spawn a dummy process
http://hg.python.org/cpython/rev/0528ec18e230
msg161592 - (view) Author: Terry J. Reedy (terry.reedy) * (Python committer) Date: 2012-05-25 16:46
3.3 commit is http://hg.python.org/cpython/rev/9373ca8c6c55
Richard, it did not record here because you just said 'Merge' rather than "#14881 merge" ;-).
Great to see a crasher fixed. Ready to close?
msg161610 - (view) Author: Richard Oudkerk (sbt) * (Python committer) Date: 2012-05-25 19:37
I'll, remember that in future;-)

Closing.
msg200302 - (view) Author: Andres Riancho (Andres.Riancho) Date: 2013-10-18 19:55
Is this a duplicate for http://bugs.python.org/issue10015
#10015 ?
msg200337 - (view) Author: Terry J. Reedy (terry.reedy) * (Python committer) Date: 2013-10-19 00:28
Since this is already fixed and closed, the question is more relevant to #10015 and whether it should be closed. The answer seems to be yes.
msg200338 - (view) Author: Terry J. Reedy (terry.reedy) * (Python committer) Date: 2013-10-19 00:33
By the way, thanks for noticing, so it *can* be closed.
History
Date User Action Args
2013-10-19 00:33:30terry.reedylinkissue10015 superseder
2013-10-19 00:33:25terry.reedysetmessages: + msg200338
2013-10-19 00:28:19terry.reedysetmessages: + msg200337
2013-10-18 19:55:52Andres.Rianchosetnosy: + Andres.Riancho
messages: + msg200302
2012-05-25 19:37:49sbtsetstatus: open -> closed
type: crash -> behavior
messages: + msg161610

resolution: fixed
stage: resolved
2012-05-25 16:46:45terry.reedysetnosy: + terry.reedy
messages: + msg161592
2012-05-25 16:38:14terry.reedysetversions: + Python 3.2, Python 3.3
2012-05-25 12:06:41python-devsetnosy: + python-dev
messages: + msg161576
2012-05-22 09:58:49sbtsetnosy: + sbt
2012-05-22 08:57:31Itay.Brandessetcomponents: + Library (Lib), - None
2012-05-22 08:51:10Itay.Brandescreate