classification
Title: allow multiple threads to efficiently send the same requests to a processing.Pool without incurring duplicate processing
Type: enhancement Stage: resolved
Components: Library (Lib) Versions: Python 3.2
process
Status: closed Resolution: postponed
Dependencies: Superseder:
Assigned To: jnoller Nosy List: DavidDecotigny, asksol, iritkatriel, jnoller
Priority: low Keywords:

Created on 2008-08-29 23:07 by DavidDecotigny, last changed 2020-12-19 00:09 by iritkatriel. This issue is now closed.

Messages (6)
msg72170 - (view) Author: David Decotigny (DavidDecotigny) Date: 2008-08-29 23:07
I posted a recipe on ASPN: http://code.activestate.com/recipes/576462/
and Jesse, cheerleader for the inclusion of (multi)processing into
python-core, suggested that it could be interesting to add this feature
to the next pythons.
This recipe is based on version 0.52 of the standalone "processing"
package, and allows to avoid redundancy when multiple threads send the
same job requests to a pool of background worker processes. The recipe
details the why and the how.
Some notes on the implementation, though:
 - There is a "Begin/End workaround" section in the code, which aims at
working around a limitation of processing 0.52 (see comments and
docstring for details). I sent issue #014431 to the issue tracker for
processing on berlios, this would allow to get rid of this workaround
 - Read my comment #2 to the recipe, dealing with my thoughts of using
weak references
msg72235 - (view) Author: Jesse Noller (jnoller) * (Python committer) Date: 2008-09-01 12:53
Thanks for adjusting the targets ben

On Aug 31, 2008, at 9:56 PM, Benjamin Peterson  
<report@bugs.python.org> wrote:

>
> Changes by Benjamin Peterson <musiccomposition@gmail.com>:
>
>
> ----------
> versions: +Python 2.7, Python 3.1 -Python 2.6, Python 3.0
>
> _______________________________________
> Python tracker <report@bugs.python.org>
> <http://bugs.python.org/issue3735>
> _______________________________________
msg72271 - (view) Author: Jesse Noller (jnoller) * (Python committer) Date: 2008-09-01 16:26
Another place this could go is in the examples FWIW
msg83259 - (view) Author: Jesse Noller (jnoller) * (Python committer) Date: 2009-03-06 23:09
Daniel, if issues are in my queue, I can manage the state
msg115081 - (view) Author: Ask Solem (asksol) (Python committer) Date: 2010-08-27 14:25
This is a nice feature, but it's also very specific and can be implemented
by extending what's already there.

Could you make a patch for this that applies to the py3k branch? If no one has the time for this, then we should probably just close the issue, until someone requests it again.
msg381152 - (view) Author: Irit Katriel (iritkatriel) * (Python triager) Date: 2020-11-16 20:27
If nobody will object in the next couple of weeks I will close this, as suggested in the previous comment.
History
Date User Action Args
2020-12-19 00:09:08iritkatrielsetstatus: pending -> closed
stage: needs patch -> resolved
2020-11-16 20:27:21iritkatrielsetstatus: open -> pending

nosy: + iritkatriel
messages: + msg381152

keywords: - needs review
2010-08-31 10:51:47asksolsetresolution: postponed
stage: test needed -> needs patch
2010-08-27 14:25:18asksolsetnosy: + asksol
messages: + msg115081
2010-08-09 04:36:01terry.reedysetversions: + Python 3.2, - Python 3.1, Python 2.7
2009-03-29 15:45:56jnollersetpriority: low
2009-03-06 23:09:11jnollersetkeywords: + needs review
messages: + msg83259
2009-03-06 21:44:19ajaksu2setassignee: jnoller
stage: test needed
2008-09-01 16:26:56jnollersetmessages: + msg72271
2008-09-01 12:53:15jnollersetmessages: + msg72235
2008-09-01 01:56:40benjamin.petersonsetversions: + Python 3.1, Python 2.7, - Python 2.6, Python 3.0
2008-08-29 23:07:08DavidDecotignycreate