classification
Title: multiprocessing.Pool is missing a starmap[_async]() method.
Type: enhancement Stage: resolved
Components: Library (Lib) Versions: Python 3.3
process
Status: closed Resolution: fixed
Dependencies: Superseder:
Assigned To: Nosy List: amaury.forgeotdarc, hynek, jnoller, neologix, pitrou, python-dev
Priority: normal Keywords: patch

Created on 2011-08-08 08:21 by hynek, last changed 2011-12-21 10:05 by pitrou. This issue is now closed.

Files
File name Uploaded Description Edit
c02fbcda56f3.diff hynek, 2011-12-20 18:56 Patch to add starmap() and starmap_async() in multiprocessing + tests + docs. review
Repositories containing patches
https://bitbucket.org/hynek/cpython-starmap#mp-starmap
Messages (11)
msg141761 - (view) Author: Hynek Schlawack (hynek) * (Python committer) Date: 2011-08-08 08:21
After I've seen a co-worker to unpack tuples while using multiprocessing.Pool.map(), I realized that the module is missing a starmap() method like itertools has.

I took it as a opportunity to contribute some code and implemented both starmap() and starmap_async().

The patch including tests is attached, please let me know, what you think. Please don't call me names, it's my first patch for such a high profile project like Python. ;)
msg141768 - (view) Author: Hynek Schlawack (hynek) * (Python committer) Date: 2011-08-08 10:52
In all my excitement, I somehow presumed that the docstring automagically lands in the docs. Added doc entries to patch (I hope they aren't too crude, I'm not a native speaker).

Same as first patch otherwise.
msg141935 - (view) Author: Amaury Forgeot d'Arc (amaury.forgeotdarc) * (Python committer) Date: 2011-08-12 02:09
+def starmapstar(args):
+    return list(itertools.starmap(args[0], args[1]))

Is your new function restricted to 2 arguments?
msg141941 - (view) Author: Hynek Schlawack (hynek) * (Python committer) Date: 2011-08-12 07:34
No, that's just a helper function like the `mapstar` directly above. args[1] is the iterable with tuples that get unpacked as arguments.
msg149914 - (view) Author: Antoine Pitrou (pitrou) * (Python committer) Date: 2011-12-20 14:58
This looks like a reasonable request to me, and the patch looks generally ok as well.
msg149916 - (view) Author: Antoine Pitrou (pitrou) * (Python committer) Date: 2011-12-20 15:03
One nit: the patch needs "versionadded" tags for the two new functions.
msg149921 - (view) Author: Hynek Schlawack (hynek) * (Python committer) Date: 2011-12-20 17:51
Thanks for the feedback Antoine!

I updated the patch to latest default tip and added the requested tags to the docs.
msg149922 - (view) Author: Charles-Fran├žois Natali (neologix) * (Python committer) Date: 2011-12-20 18:18
Looks good to me, except for another minor nit:
"""
the elements of the `iterable` are expected to be tuples
"""

AFAICT, you just require the elements of `├Čterables` to be iterables, not necessarily tuples.
msg149923 - (view) Author: Hynek Schlawack (hynek) * (Python committer) Date: 2011-12-20 18:56
You're right. I've updated the docs accordingly, thanks!
msg149975 - (view) Author: Roundup Robot (python-dev) Date: 2011-12-21 10:04
New changeset b07b1e58582d by Antoine Pitrou in branch 'default':
Issue #12708: Add starmap() and starmap_async() methods (similar to itertools.starmap()) to multiprocessing.Pool.
http://hg.python.org/cpython/rev/b07b1e58582d
msg149976 - (view) Author: Antoine Pitrou (pitrou) * (Python committer) Date: 2011-12-21 10:05
Patch committed. Thanks for your contribution!
History
Date User Action Args
2011-12-21 10:05:02pitrousetstatus: open -> closed
resolution: fixed
messages: + msg149976

stage: patch review -> resolved
2011-12-21 10:04:30python-devsetnosy: + python-dev
messages: + msg149975
2011-12-20 18:56:57hyneksetfiles: - 8a9e14cda106.diff
2011-12-20 18:56:23hyneksetfiles: + c02fbcda56f3.diff
2011-12-20 18:56:05hyneksetmessages: + msg149923
2011-12-20 18:18:32neologixsetmessages: + msg149922
2011-12-20 17:54:08hyneksetfiles: - mp-starmap-w-docs.diff
2011-12-20 17:51:56hyneksetfiles: + 8a9e14cda106.diff
2011-12-20 17:51:10hyneksethgrepos: + hgrepo97
messages: + msg149921
2011-12-20 15:03:09pitrousetmessages: + msg149916
2011-12-20 14:58:45pitrousetversions: - Python 2.6, Python 3.1, Python 2.7, Python 3.2, Python 3.4
nosy: + pitrou, neologix

messages: + msg149914

stage: patch review
2011-08-12 07:34:41hyneksetmessages: + msg141941
2011-08-12 02:09:11amaury.forgeotdarcsetnosy: + amaury.forgeotdarc
messages: + msg141935
2011-08-08 11:12:46hyneksetfiles: - mp-starmap.diff
2011-08-08 10:52:06hyneksetfiles: + mp-starmap-w-docs.diff

messages: + msg141768
2011-08-08 08:21:45hynekcreate