classification
Title: Documentation for multiprocessing - Pool.apply()
Type: Stage: needs patch
Components: Documentation Versions: Python 3.0, Python 2.7, Python 2.6
process
Status: closed Resolution: fixed
Dependencies: Superseder:
Assigned To: jnoller Nosy List: beazley, georg.brandl, jnoller
Priority: normal Keywords: easy

Created on 2008-12-08 15:45 by beazley, last changed 2009-01-22 22:06 by jnoller. This issue is now closed.

Messages (4)
msg77312 - (view) Author: David M. Beazley (beazley) Date: 2008-12-08 15:45
The documentation for the apply() and apply_async() methods of a Pool 
object might emphasize that these operations execute func(*args,**kwargs) 
in only one of the pool workers and that func() is not being executed in 
parallel on all workers.    On first reading, I actually thought it might 
be the latter (and had to do some experimentation to figure out what was 
actually happening).
msg77316 - (view) Author: Jesse Noller (jnoller) * (Python committer) Date: 2008-12-08 16:11
Ah, I see what you're talking about David. Good catch. It wouldn't make 
sense to run the apply() in all of the workers, as apply() is a single 
function(...) call. map() however is done in parallel amongst all workers.
msg77336 - (view) Author: David M. Beazley (beazley) Date: 2008-12-08 19:11
Actually, you shouldn't discount the potential usefulness of running 
apply() in all of the worker nodes.  A lot of people coming from 
parallel programming know about things like global broadcasts, 
reductions, and so forth.  For example, if I wanted to perform a global 
operation (maybe some kind of configuration) on all workers, I could see 
doing some kind of global apply() operation to do it.  

That said, I'm not actually asking for any new functionality.   I'd just 
make it more clear that apply() is not performing a function call on all 
pool workers.

Also, given that apply() blocks, I'm not exactly sure how useful it is 
in the context of actually performing work in parallel.   You might want 
to emphasize that apply_async() is better suited for that (the only 
other way I could think of to take advantage of apply() in parallel 
would be to call it from separate threads in the process that created 
the pool).
msg80385 - (view) Author: Jesse Noller (jnoller) * (Python committer) Date: 2009-01-22 22:06
fixed in trunk, r68862, merged to 3k in r68863
History
Date User Action Args
2009-01-22 22:06:46jnollersetstatus: open -> closed
resolution: fixed
messages: + msg80385
2008-12-08 19:11:41beazleysetmessages: + msg77336
2008-12-08 16:11:35jnollersetmessages: + msg77316
2008-12-08 15:50:59christian.heimessetpriority: normal
assignee: georg.brandl -> jnoller
stage: needs patch
keywords: + easy
nosy: + jnoller
2008-12-08 15:45:19beazleycreate