This issue tracker has been migrated to GitHub, and is currently read-only.
For more information, see the GitHub FAQs in the Python's Developer Guide.

Title: multiprocessing.Pool.imap swallows exceptions thrown by generators
Type: behavior Stage: resolved
Components: Library (Lib) Versions: Python 3.6, Python 3.5
Status: closed Resolution: duplicate
Dependencies: Superseder: Imap from ThreadPool behaves unexpectedly
View: 28699
Assigned To: Nosy List: davin, elias
Priority: normal Keywords:

Created on 2016-11-06 16:16 by elias, last changed 2022-04-11 14:58 by admin. This issue is now closed.

Messages (4)
msg280146 - (view) Author: Elias Zamaria (elias) * Date: 2016-11-06 16:16
I have the following code:

from multiprocessing import Pool

def double(x):
    return 2 * x

def get_numbers():
    raise Exception("oops")
    yield 1
    yield 2

print(list(Pool(processes=2).imap(double, get_numbers())))

I would expect it to raise an exception, but instead it just prints "[]", seeming to indicate that imap ran fine and produced no values.

This seems similar to the behavior described in bugs 23051 and 26333, but this happens if the iterator directly raises an exception before yielding anything. If I move the raise statement below one or both of the yields, I get an exception from imap as expected.

I am experiencing this with Python 3.5.2 on OS X 10.11.6. I haven't tried it with any other versions.
msg280913 - (view) Author: Elias Zamaria (elias) * Date: 2016-11-16 05:29
I tried my code in Python 3.6.0b3 and got the same result.
msg280957 - (view) Author: Davin Potts (davin) * (Python committer) Date: 2016-11-16 14:48
In issue28699, an important clue has been identified (a variation on the generator triggers a different, inconsistent behavior).  I will merge this issue into that one to keep us all on the same page.
msg281744 - (view) Author: Davin Potts (davin) * (Python committer) Date: 2016-11-25 22:07
Closing this issue -- all further discussion moves to issue28699
Date User Action Args
2022-04-11 14:58:39adminsetgithub: 72811
2016-11-25 22:07:05davinsetstatus: open -> closed
resolution: duplicate
messages: + msg281744

stage: resolved
2016-11-16 14:48:38davinsetsuperseder: Imap from ThreadPool behaves unexpectedly
messages: + msg280957
2016-11-16 06:52:40serhiy.storchakasetnosy: + davin
2016-11-16 05:29:56eliassetmessages: + msg280913
versions: + Python 3.6
2016-11-06 16:16:43eliascreate