This issue tracker has been migrated to GitHub, and is currently read-only.
For more information, see the GitHub FAQs in the Python's Developer Guide.

Author xiang.zhang
Recipients davin, elias, lev-veshnyakov, xiang.zhang
Date 2016-11-28.08:22:21
SpamBayes Score -1.0
Marked as misclassified Yes
Message-id <1480321341.87.0.5422627059.issue28699@psf.upfronthosting.co.za>
In-reply-to
Content
> 4.  Guard against misbehaving generators/iterables *before* they are put into the taskqueue.

This approach is good. 2 points about the patch:

1. How about _map_async(map)? Does it need the same strategy? For an iterator with __len__ defined it seems to get the same issue as here.

from multiprocessing import Pool
def double(x):
    return 2 * x
class buggy:
        def __iter__(self):
                return self
        def __next__(self):
                raise Exception('oops')
        def __len__(self):
                return 1
list(Pool(processes=2).map(double, buggy()))
list(Pool(processes=2).map(double, buggy()))  # hangs

2. The logic in _handle_tasks to handle task, i to could be removed I think. With _guarded_task_generation the for loop cannot fail and the logic itself is buggy now.
History
Date User Action Args
2016-11-28 08:22:21xiang.zhangsetrecipients: + xiang.zhang, elias, davin, lev-veshnyakov
2016-11-28 08:22:21xiang.zhangsetmessageid: <1480321341.87.0.5422627059.issue28699@psf.upfronthosting.co.za>
2016-11-28 08:22:21xiang.zhanglinkissue28699 messages
2016-11-28 08:22:21xiang.zhangcreate