classification
Title: allow multiple threads to efficiently send the same requests to a processing.Pool without incurring duplicate processing
Type: enhancement Stage: needs patch
Components: Library (Lib) Versions: Python 3.2
process
Status: open Resolution: postponed
Dependencies: Superseder:
Assigned To: jnoller Nosy List: DavidDecotigny, asksol, jnoller
Priority: low Keywords: needs review

Created on 2008-08-29 23:07 by DavidDecotigny, last changed 2010-08-31 10:51 by asksol.

Messages (5)
msg72170 - (view) Author: David Decotigny (DavidDecotigny) Date: 2008-08-29 23:07
I posted a recipe on ASPN: http://code.activestate.com/recipes/576462/
and Jesse, cheerleader for the inclusion of (multi)processing into
python-core, suggested that it could be interesting to add this feature
to the next pythons.
This recipe is based on version 0.52 of the standalone "processing"
package, and allows to avoid redundancy when multiple threads send the
same job requests to a pool of background worker processes. The recipe
details the why and the how.
Some notes on the implementation, though:
 - There is a "Begin/End workaround" section in the code, which aims at
working around a limitation of processing 0.52 (see comments and
docstring for details). I sent issue #014431 to the issue tracker for
processing on berlios, this would allow to get rid of this workaround
 - Read my comment #2 to the recipe, dealing with my thoughts of using
weak references
msg72235 - (view) Author: Jesse Noller (jnoller) * (Python committer) Date: 2008-09-01 12:53
Thanks for adjusting the targets ben

On Aug 31, 2008, at 9:56 PM, Benjamin Peterson  
<report@bugs.python.org> wrote:

>
> Changes by Benjamin Peterson <musiccomposition@gmail.com>:
>
>
> ----------
> versions: +Python 2.7, Python 3.1 -Python 2.6, Python 3.0
>
> _______________________________________
> Python tracker <report@bugs.python.org>
> <http://bugs.python.org/issue3735>
> _______________________________________
msg72271 - (view) Author: Jesse Noller (jnoller) * (Python committer) Date: 2008-09-01 16:26
Another place this could go is in the examples FWIW
msg83259 - (view) Author: Jesse Noller (jnoller) * (Python committer) Date: 2009-03-06 23:09
Daniel, if issues are in my queue, I can manage the state
msg115081 - (view) Author: Ask Solem (asksol) (Python committer) Date: 2010-08-27 14:25
This is a nice feature, but it's also very specific and can be implemented
by extending what's already there.

Could you make a patch for this that applies to the py3k branch? If no one has the time for this, then we should probably just close the issue, until someone requests it again.
History
Date User Action Args
2010-08-31 10:51:47asksolsetresolution: postponed
stage: test needed -> needs patch
2010-08-27 14:25:18asksolsetnosy: + asksol
messages: + msg115081
2010-08-09 04:36:01terry.reedysetversions: + Python 3.2, - Python 3.1, Python 2.7
2009-03-29 15:45:56jnollersetpriority: low
2009-03-06 23:09:11jnollersetkeywords: + needs review
messages: + msg83259
2009-03-06 21:44:19ajaksu2setassignee: jnoller
stage: test needed
2008-09-01 16:26:56jnollersetmessages: + msg72271
2008-09-01 12:53:15jnollersetmessages: + msg72235
2008-09-01 01:56:40benjamin.petersonsetversions: + Python 3.1, Python 2.7, - Python 2.6, Python 3.0
2008-08-29 23:07:08DavidDecotignycreate