This issue tracker has been migrated to GitHub, and is currently read-only.
For more information, see the GitHub FAQs in the Python's Developer Guide.

Author dlukes
Recipients dlukes, docs@python, pitrou
Date 2017-12-20.12:06:40
SpamBayes Score -1.0
Marked as misclassified Yes
Message-id <1513771600.8.0.213398074469.issue32306@psf.upfronthosting.co.za>
In-reply-to
Content
Yes, sorry for not being quite clear the first time around :)

I eventually found out about Pool.imap (see item 3 on list in OP) and indeed it fits my use case very nicely, but my point was that the documentation is somewhat misleading with respect to the semantics of built-in `map()` in Python 3.

Specifically, I would argue that it is unexpected for a function which claims to be "Equivalent to map(func, *iterables)" to require allocating a list the length of the shortest iterable.

Maybe a code example will make this clearer for potential newcomers to the discussion -- this is what I would expect to happen (= the behavior of built-in `map()` itself), yielding values from the iterable is interleaved with calls to the mapped function:

```
>>> def gen():
...     for i in range(3):
...         print("yielding", i)
...         yield i
...         
>>> def add1(i):
...     print("adding 1 to", i)
...     return i + 1
... 
>>> list(map(add1, gen()))
yielding 0
adding 1 to 0
yielding 1
adding 1 to 1
yielding 2
adding 1 to 2
[1, 2, 3]
```

This is what happens instead with `concurrent.futures.Executor.map()`:

```
>>> def my_map(fn, iterable):
...     lst = list(iterable)
...     for i in lst:
...         yield fn(i)
...         
>>> list(my_map(add1, gen()))
yielding 0
yielding 1
yielding 2
adding 1 to 0
adding 1 to 1
adding 1 to 2
[1, 2, 3]
```
History
Date User Action Args
2017-12-20 12:06:40dlukessetrecipients: + dlukes, pitrou, docs@python
2017-12-20 12:06:40dlukessetmessageid: <1513771600.8.0.213398074469.issue32306@psf.upfronthosting.co.za>
2017-12-20 12:06:40dlukeslinkissue32306 messages
2017-12-20 12:06:40dlukescreate