Author Klamann
Recipients Klamann, xiang.zhang
Date 2017-05-12.09:55:30
SpamBayes Score -1.0
Marked as misclassified Yes
Message-id <1494582931.29.0.123591829804.issue30323@psf.upfronthosting.co.za>
In-reply-to
Content
Yes, I was wrong in my assumption that simply replacing the list comprehension with a generator expression would fix the issue.

Nevertheless, there is no need to load the *entire* generator into memory by converting it to a list. All we have to read are the first n elements, where n is the number of workers that are currently available.

I've implemented an alternative solution that works for me, using wait() and notify() from threading.Condition, but I'm not quite sure if this would be the best solution for everyone. But I could post it here, if you're intrested.

We should also consider that not strictly evaluating every iterable that is passed to the map() function might break existing code that implicitly relies on that fact that this is happening (although this is not a documented feature of the map function and was probably not the intended behaviour in the first place).
History
Date User Action Args
2017-05-12 09:55:31Klamannsetrecipients: + Klamann, xiang.zhang
2017-05-12 09:55:31Klamannsetmessageid: <1494582931.29.0.123591829804.issue30323@psf.upfronthosting.co.za>
2017-05-12 09:55:31Klamannlinkissue30323 messages
2017-05-12 09:55:30Klamanncreate