This issue tracker has been migrated to GitHub, and is currently read-only.
For more information, see the GitHub FAQs in the Python's Developer Guide.

classification
Title: Multiprocessing hangs when multiprocessing.Pool methods are called
Type: crash Stage:
Components: Library (Lib) Versions: Python 3.0, Python 2.6
process
Status: closed Resolution: fixed
Dependencies: Superseder:
Assigned To: jnoller Nosy List: jnoller, mishok13, roudkerk
Priority: normal Keywords:

Created on 2008-07-03 16:48 by mishok13, last changed 2022-04-11 14:56 by admin. This issue is now closed.

Messages (2)
msg69207 - (view) Author: Andrii V. Mishkovskyi (mishok13) Date: 2008-07-03 16:48
`multiprocessing` hangs, when `multiprocessing.Pool` methods `map`,
`imap`, `imap_unordered`, `apply`, `apply_async`, `map_async` are called.
Here is an example:

Python 3.0b1+ (py3k:64686M, Jul  3 2008, 13:06:13)
[GCC 4.2.3 (Ubuntu 4.2.3-2ubuntu7)] on linux2
Type "help", "copyright", "credits" or "license" for more information.
>>> from multiprocessing import Pool
>>> Pool(processes=4).imap(lambda x: x, range(10))
<multiprocessing.pool.IMapIterator object at 0x83bcc8c>
>>> Pool(processes=4).map(lambda x: x, range(10))
Exception in thread Thread-13:
Traceback (most recent call last):
  File "/usr/local/lib/python3.0/threading.py", line 492, in
_bootstrap_inner
    self.run()
  File "/usr/local/lib/python3.0/threading.py", line 447, in run
    self._target(*self._args, **self._kwargs)
  File "/usr/local/lib/python3.0/multiprocessing/pool.py", line 225, in
_handle_tasks
    put(task)
  File "/usr/local/lib/python3.0/pickle.py", line 1319, in dumps
    Pickler(f, protocol).dump(obj)
  File "/usr/local/lib/python3.0/multiprocessing/util.py", line 311, in
_reduce_method
    if m.__self__ is None:
AttributeError: 'function' object has no attribute '__self__'

Process PoolWorker-28:
Traceback (most recent call last):
  File "/usr/local/lib/python3.0/multiprocessing/process.py", line 232,
in _bootstrap
    self.run()
  File "/usr/local/lib/python3.0/multiprocessing/process.py", line 88,
in run
    self._target(*self._args, **self._kwargs)
  File "/usr/local/lib/python3.0/multiprocessing/pool.py", line 57, in
worker
    task = get()
  File "/usr/local/lib/python3.0/multiprocessing/queues.py", line 337,
in get
Process PoolWorker-27:
Traceback (most recent call last):
  File "/usr/local/lib/python3.0/multiprocessing/process.py", line 232,
in _bootstrap
    self.run()
  File "/usr/local/lib/python3.0/multiprocessing/process.py", line 88,
in run
    self._target(*self._args, **self._kwargs)
  File "/usr/local/lib/python3.0/multiprocessing/pool.py", line 57, in
worker
    task = get()
  File "/usr/local/lib/python3.0/multiprocessing/queues.py", line 337,
in get
Process PoolWorker-26:
Traceback (most recent call last):
  File "/usr/local/lib/python3.0/multiprocessing/process.py", line 232,
in _bootstrap
Process PoolWorker-25:
Traceback (most recent call last):
  File "/usr/local/lib/python3.0/multiprocessing/process.py", line 232,
in _bootstrap
    self.run()
  File "/usr/local/lib/python3.0/multiprocessing/process.py", line 88,
in run
    self._target(*self._args, **self._kwargs)
  File "/usr/local/lib/python3.0/multiprocessing/pool.py", line 57, in
worker
    task = get()
  File "/usr/local/lib/python3.0/multiprocessing/queues.py", line 337,
in get
    self.run()
  File "/usr/local/lib/python3.0/multiprocessing/process.py", line 88,
in run
    self._target(*self._args, **self._kwargs)
  File "/usr/local/lib/python3.0/multiprocessing/pool.py", line 57, in
worker
    task = get()
  File "/usr/local/lib/python3.0/multiprocessing/queues.py", line 339,
in get
    return recv()
KeyboardInterrupt
    racquire()
KeyboardInterrupt
    racquire()
KeyboardInterrupt
    racquire()
KeyboardInterrupt
Terminated

The same happens on latest trunk of Python 2.6.
Note, `map` and `apply` methods hang Python interpreter completely, so I
have to use `kill` utility to exit Python.
msg80375 - (view) Author: Jesse Noller (jnoller) * (Python committer) Date: 2009-01-22 19:25
This is now covered in the multiprocessing documentation:

"Note Functionality within this package requires that the __main__ method 
be importable by the children. This is covered in Programming guidelines 
however it is worth pointing out here. This means that some examples, such 
as the multiprocessing.Pool examples will not work in the interactive 
interpreter."
History
Date User Action Args
2022-04-11 14:56:36adminsetgithub: 47522
2009-01-22 19:25:51jnollersetstatus: open -> closed
resolution: fixed
messages: + msg80375
2009-01-19 16:42:08jnollersetassignee: jnoller
2008-07-03 16:48:37mishok13create