This issue tracker has been migrated to GitHub, and is currently read-only.
For more information, see the GitHub FAQs in the Python's Developer Guide.

Title: Creating a multiprocess.pool.ThreadPool from a child thread blows up.
Type: behavior Stage: resolved
Components: Library (Lib) Versions: Python 2.7
Status: closed Resolution: duplicate
Dependencies: Superseder: multiprocessing.dummy craches when self._parent._children does not exist
View: 14881
Assigned To: Nosy List: Michael.Olson, acordiner, asksol, chris-, flox, jnoller, sbt, terry.reedy
Priority: normal Keywords:

Created on 2010-10-02 14:00 by Michael.Olson, last changed 2022-04-11 14:57 by admin. This issue is now closed.

File name Uploaded Description Edit Michael.Olson, 2010-10-02 14:00 Demonstration code
Messages (5)
msg117875 - (view) Author: Michael Olson (Michael.Olson) Date: 2010-10-02 14:00
Using Python 2.7 x32 on Windows XP

Attempting to create a multiprocessing.pool.ThreadPool
in a child thread created using threading.Thread, an
AttributeError is thrown. A ThreadPool created in the 
main thread can be passed to the child thread and used.

Exact text of exception
  File "D:\Dev\Python27\lib\multiprocessing\dummy\", line 47, in star
    self._parent._children[self] = None
AttributeError: 'Thread' object has no attribute '_children'

Demonstration Code
import unittest
from threading import Thread
from multiprocessing.pool import ThreadPool

def f(x):
    return x*x

def create_and_run(cb, pool = None):
    if not pool:
        pool = ThreadPool(2)
    r = pool.map_async(f, range(10))

class TestThreadPool(unittest.TestCase):
    def setUp(self):
        self.expected = [f(x) for x in range(10)]

    def callback(self, data): = data

    def test_creating_pool_in_mainthread(self):
        """Test multiprocessing.pool.ThreadPool from main thread""" = None
        self.assertEqual(, self.expected)

    def test_creating_pool_in_subthread(self):
        """Test multiprocessing.pool.ThreadPool from a child thread.""" = None
        t = Thread(target=create_and_run, args=[self.callback])
        self.assertEqual(, self.expected)

    def test_creating_pool_in_subthread_workaround(self):
        """Test running a ThreadPool created in main thread, used in child.""" = None
        pool = ThreadPool(2)
        t = Thread(target=create_and_run, args=[self.callback, pool])
        self.assertEqual(, self.expected)

if __name__ =='__main__':
    suite = unittest.TestLoader().loadTestsFromTestCase(TestThreadPool)
msg117890 - (view) Author: Jesse Noller (jnoller) * (Python committer) Date: 2010-10-02 17:15
I can not, for the life of me, remember why ThreadPool is there, except as a fallback. It's also not part of the documented interface as well. Additionally, in Python 3 we now have futures.
msg142294 - (view) Author: Christian (chris-) Date: 2011-08-17 22:42
A workaround would be to call the following in the thread you want to use ThreadPool:

if not hasattr(threading.current_process(), "_children"):
    threading.current_process()._children = weakref.WeakKeyDictionary()

(putting this in Process could be a very simple patch)
msg146473 - (view) Author: Alister Cordiner (acordiner) * Date: 2011-10-27 05:47
I think that workaround should be:

if not hasattr(threading.current_thread(), "_children"):
    threading.current_thread()._children = weakref.WeakKeyDictionary()
msg200339 - (view) Author: Terry J. Reedy (terry.reedy) * (Python committer) Date: 2013-10-19 00:33
#14881 seems to be a duplicate of this. It was closed in May 2012 after 2.7, 3.2, and 3.3 were patched. The three tests in now pass with 2.7.5 and 3.3.2. So closing.
Date User Action Args
2022-04-11 14:57:07adminsetgithub: 54224
2013-10-19 00:33:30terry.reedysetstatus: open -> closed

superseder: multiprocessing.dummy craches when self._parent._children does not exist

nosy: + terry.reedy
messages: + msg200339
resolution: duplicate
stage: resolved
2013-10-18 20:33:00sbtsetnosy: + sbt
2011-10-27 09:45:33floxsetnosy: + flox
2011-10-27 05:47:33acordinersetnosy: + acordiner
messages: + msg146473
2011-08-27 14:58:59vinay.sajipsettitle: Creating a multiproccess.pool.ThreadPool from a child thread blows up. -> Creating a multiprocess.pool.ThreadPool from a child thread blows up.
2011-08-17 22:42:10chris-setnosy: + chris-
messages: + msg142294
2010-10-02 17:15:15jnollersetmessages: + msg117890
2010-10-02 14:49:08r.david.murraysetnosy: + jnoller, asksol
type: crash -> behavior
2010-10-02 14:00:28Michael.Olsoncreate