msg273741 - (view) |
Author: Jason Yu (Jason Yu) |
Date: 2016-08-26 23:15 |
Hello all,
The official doc describes `Pool.map(func, iterable[, chunksize])` as "A parallel equivalent of the map() built-in function.".
Since the function signature of `map` is `map(function, iterable, ...)`, I hereby suggest that `Pool.map` should change its function signature to `Pool.map(function, iterable, ... [, chunksize])`. This will bring true equivalent to these functions.
Tell me what you think.
Pool.map: https://docs.python.org/3/library/multiprocessing.html#multiprocessing.pool.Pool.map
map: https://docs.python.org/3/library/functions.html#map
Jason
|
msg273742 - (view) |
Author: Raymond Hettinger (rhettinger) * |
Date: 2016-08-27 00:48 |
-0 We need better motivation than "that is was map() does". Instead, are should look at whether there are compelling use cases and its impact on the complexity of the multiprocessing implementation. I usually just prezip my data and it works fine.
|
msg273743 - (view) |
Author: Tim Peters (tim.peters) * |
Date: 2016-08-27 00:59 |
Note that `Pool` grew `starmap()` and `starmap_async()` methods in Python 3.3 to (mostly) address this.
The signature difference from the old builtin `map()` remains regrettable. Note that the `Pool` version differs from the `concurrent.futures` version of `map()` in this respect too.
|
msg277478 - (view) |
Author: (naught101) |
Date: 2016-09-27 00:04 |
It would be helpful if the docs at https://docs.python.org/3/library/multiprocessing.html#multiprocessing.pool.Pool.map and in `help(Pool.map)` mentioned starmap in a "see also" section at the bottom.
|
msg277535 - (view) |
Author: Berker Peksag (berker.peksag) * |
Date: 2016-09-27 18:19 |
I think we can add a note for starmap() in the following sentence:
> [...] (it supports only one iterable argument though).
(Quoted from https://docs.python.org/3/library/multiprocessing.html#multiprocessing.pool.Pool.map)
Would you like to write a patch?
|
msg277572 - (view) |
Author: (naught101) |
Date: 2016-09-28 02:22 |
Happy to. What is the process? hg clone cpython, and make a patch from that and post it here? Or is there some pull-request style process available?
|
msg277573 - (view) |
Author: Berker Peksag (berker.peksag) * |
Date: 2016-09-28 02:30 |
> What is the process? hg clone cpython, and make a patch from that and post it here?
Yes, it's exactly what you just described :) See https://docs.python.org/devguide/index.html for details.
> Or is there some pull-request style process available?
Not yet, but we will move to GitHub soon.
|
msg277574 - (view) |
Author: (naught101) |
Date: 2016-09-28 02:35 |
OK, one question, is Pool.map preferable to Pool.starmap in any particular cases?
|
msg277576 - (view) |
Author: (naught101) |
Date: 2016-09-28 02:49 |
Here you go.
|
msg277680 - (view) |
Author: Davin Potts (davin) * |
Date: 2016-09-29 02:21 |
@naught101: Thanks for the wording change -- I think your suggested doc change will be a good one. I've added a comment which you can access via the "review" link to the right of the link for your mp.map.starmap.patch attachment.
|
msg348177 - (view) |
Author: Flavian Hautbois (flavianhautbois) * |
Date: 2019-07-19 13:56 |
Are you going to open a PR for this patch? I think that would be a nice addition
|
msg352973 - (view) |
Author: Ido Michael (Ido Michael) * |
Date: 2019-09-22 12:09 |
Hey,
Is it still open?
What else needs to be done?
Ido
|
msg355203 - (view) |
Author: Kyle Stanley (aeros) * |
Date: 2019-10-23 01:33 |
> Is it still open? What else needs to be done?
Yes, this patch needs to be translated into a GitHub PR. See https://devguide.python.org/pullrequest/ for more information on our PR workflow if you're not already familiar with it. Since naught101 wrote a patch, we should have their name in "Co-Authored-By" of the final commit to the PR if they have a GitHub account. If not, just mention it in a comment to the PR that they originally wrote the patch.
Feel free to @mention me when you open the PR so I can help to review it, although this will likely require a final approval from a maintainer of multiprocessing module before it can be merged.
|
msg357769 - (view) |
Author: miss-islington (miss-islington) |
Date: 2019-12-03 23:31 |
New changeset eb48a451e3844185b9a8751c9badffbddc89689d by Miss Islington (bot) (An Long) in branch 'master':
bpo-27873: Update docstring for multiprocessing.Pool.map (GH-17436)
https://github.com/python/cpython/commit/eb48a451e3844185b9a8751c9badffbddc89689d
|
msg357770 - (view) |
Author: miss-islington (miss-islington) |
Date: 2019-12-03 23:36 |
New changeset 55a7046471e19843a68d4a1a15252fd197bb6913 by Miss Islington (bot) in branch '3.7':
bpo-27873: Update docstring for multiprocessing.Pool.map (GH-17436)
https://github.com/python/cpython/commit/55a7046471e19843a68d4a1a15252fd197bb6913
|
msg357771 - (view) |
Author: miss-islington (miss-islington) |
Date: 2019-12-03 23:37 |
New changeset baf07395eaa77e515ddfa1d3f42785d50b4d2889 by Miss Islington (bot) in branch '3.8':
bpo-27873: Update docstring for multiprocessing.Pool.map (GH-17436)
https://github.com/python/cpython/commit/baf07395eaa77e515ddfa1d3f42785d50b4d2889
|
msg362085 - (view) |
Author: Ido Michael (Ido Michael) * |
Date: 2020-02-16 17:22 |
Can this issue be closed, I see it was merged successfully?
|
msg362112 - (view) |
Author: Kyle Stanley (aeros) * |
Date: 2020-02-16 23:15 |
> Can this issue be closed, I see it was merged successfully?
Yep, I'll go ahead and close the issue now. Thanks for the reminder; I got caught up in a few other projects.
|
|
Date |
User |
Action |
Args |
2022-04-11 14:58:35 | admin | set | github: 72060 |
2020-02-16 23:15:41 | aeros | set | status: open -> closed
messages:
+ msg362112 stage: patch review -> resolved |
2020-02-16 17:22:25 | Ido Michael | set | messages:
+ msg362085 |
2019-12-03 23:37:44 | miss-islington | set | messages:
+ msg357771 |
2019-12-03 23:36:51 | miss-islington | set | messages:
+ msg357770 |
2019-12-03 23:31:15 | miss-islington | set | pull_requests:
+ pull_request16940 |
2019-12-03 23:31:08 | miss-islington | set | pull_requests:
+ pull_request16939 |
2019-12-03 23:31:07 | miss-islington | set | nosy:
+ miss-islington messages:
+ msg357769
|
2019-12-02 15:03:09 | asaka | set | pull_requests:
+ pull_request16916 |
2019-11-24 14:05:36 | asaka | set | stage: needs patch -> patch review pull_requests:
+ pull_request16853 |
2019-10-23 01:33:34 | aeros | set | nosy:
+ aeros messages:
+ msg355203
|
2019-09-22 12:09:43 | Ido Michael | set | nosy:
+ Ido Michael messages:
+ msg352973
|
2019-07-19 13:56:00 | flavianhautbois | set | nosy:
+ flavianhautbois messages:
+ msg348177
|
2016-09-29 02:21:24 | davin | set | messages:
+ msg277680 |
2016-09-28 02:49:26 | naught101 | set | files:
+ mp.map.starmap.patch keywords:
+ patch messages:
+ msg277576
|
2016-09-28 02:35:18 | naught101 | set | messages:
+ msg277574 |
2016-09-28 02:30:15 | berker.peksag | set | messages:
+ msg277573 |
2016-09-28 02:22:30 | naught101 | set | messages:
+ msg277572 |
2016-09-27 18:19:41 | berker.peksag | set | components:
+ Documentation, - Interpreter Core versions:
+ Python 3.5, Python 3.7 keywords:
+ easy nosy:
+ berker.peksag
messages:
+ msg277535 stage: needs patch |
2016-09-27 00:04:21 | naught101 | set | nosy:
+ naught101 messages:
+ msg277478
|
2016-08-27 00:59:28 | tim.peters | set | nosy:
+ tim.peters messages:
+ msg273743
|
2016-08-27 00:48:26 | rhettinger | set | assignee: davin
messages:
+ msg273742 nosy:
+ rhettinger, davin |
2016-08-26 23:15:50 | Jason Yu | create | |