This issue tracker has been migrated to GitHub, and is currently read-only.
For more information, see the GitHub FAQs in the Python's Developer Guide.

Author Aaron Halfaker
Recipients Aaron Halfaker
Date 2016-02-10.19:03:45
SpamBayes Score -1.0
Marked as misclassified Yes
Message-id <1455131025.64.0.640942257663.issue26333@psf.upfronthosting.co.za>
In-reply-to
Content
multiprocessing.imap will hang and not raise an error if an error occurs in the generator that is being mapped over.  I'd expect the error to be raised and/or the process to fail.  

For example, run the following code in python 2.7 or 3.4:

    from multiprocessing import Pool

    def add_one(v):
        return v+1

    pool = Pool(processes=2)

    values = ["1", "2", "3", "4", "foo", "5", "6", "7", "8"]
    value_iter = (int(v) for v in values)

    for new_val in pool.imap(add_one, value_iter):
        print(new_val)

And output should look something like this:

    $ python demo_hanging.py 
    2
    3
    4
    5
    Exception in thread Thread-2:
    Traceback (most recent call last):
      File "/usr/lib/python3.4/threading.py", line 920, in _bootstrap_inner
        self.run()
      File "/usr/lib/python3.4/threading.py", line 868, in run
        self._target(*self._args, **self._kwargs)
      File "/usr/lib/python3.4/multiprocessing/pool.py", line 378, in _handle_tasks
        for i, task in enumerate(taskseq):
      File "/usr/lib/python3.4/multiprocessing/pool.py", line 286, in <genexpr>
        self._taskqueue.put((((result._job, i, func, (x,), {})
      File "demo_hanging.py", line 9, in <genexpr>
        value_iter = (int(v) for v in values)
    ValueError: invalid literal for int() with base 10: 'foo'

The script will then hang indefinitely.
History
Date User Action Args
2016-02-10 19:03:45Aaron Halfakersetrecipients: + Aaron Halfaker
2016-02-10 19:03:45Aaron Halfakersetmessageid: <1455131025.64.0.640942257663.issue26333@psf.upfronthosting.co.za>
2016-02-10 19:03:45Aaron Halfakerlinkissue26333 messages
2016-02-10 19:03:45Aaron Halfakercreate