classification
Title: Rewrite PipeConnection and Connection in pure Python
Type: enhancement Stage: resolved
Components: Library (Lib) Versions: Python 3.3
process
Status: closed Resolution: fixed
Dependencies: Superseder:
Assigned To: Nosy List: alex, asksol, brian.curtin, jnoller, nirai, pitrou, python-dev, s7v7nislands, tim.golden
Priority: normal Keywords: patch

Created on 2011-04-02 21:33 by pitrou, last changed 2011-05-10 02:30 by nirai. This issue is now closed.

Files
File name Uploaded Description Edit
mpconn.patch pitrou, 2011-04-02 21:32 review
pipebench.py pitrou, 2011-04-03 16:03
mpconn2.patch pitrou, 2011-05-07 16:53 review
Messages (8)
msg132816 - (view) Author: Antoine Pitrou (pitrou) * (Python committer) Date: 2011-04-02 21:32
Here is a rewrite of multiprocessing.{PipeConnection,Connection} in pure Python, for ease of maintenance and improvement.
msg132835 - (view) Author: Jesse Noller (jnoller) * (Python committer) Date: 2011-04-03 12:41
Nothing jumps out at me at initial review; I've asked other contributors/interested parties to take a look too. Thanks a ton Antoine for doing this work
msg132846 - (view) Author: Antoine Pitrou (pitrou) * (Python committer) Date: 2011-04-03 16:02
The patch slows down Pipe() a bit, by the way. On my machine, each message sent and received has an additional 10µs overhead. See attached benchmark script.
msg132848 - (view) Author: Antoine Pitrou (pitrou) * (Python committer) Date: 2011-04-03 16:03
Woops, sorry for the duplicates...
msg132881 - (view) Author: Ask Solem (asksol) (Python committer) Date: 2011-04-03 20:51
This is great!  I always wondered if it was really necessary to use C for this. 10µs overhead should be worth it ;)

I've read the patch, but not carefully.  So far nothing jumps at me either.

Cheers!
msg135480 - (view) Author: Antoine Pitrou (pitrou) * (Python committer) Date: 2011-05-07 16:53
Updated patch, fixing a sporadic failure in test_spawn_close.
msg135595 - (view) Author: Roundup Robot (python-dev) Date: 2011-05-09 15:10
New changeset 1ac03e071d65 by Antoine Pitrou in branch 'default':
Issue #11743: Rewrite multiprocessing connection classes in pure Python.
http://hg.python.org/cpython/rev/1ac03e071d65
msg135597 - (view) Author: Antoine Pitrou (pitrou) * (Python committer) Date: 2011-05-09 15:13
Now checked in, hopefully there will be no buildbot failures.
History
Date User Action Args
2011-05-10 02:30:51niraisetnosy: + nirai
2011-05-09 15:49:44pitrousetstatus: pending -> closed
2011-05-09 15:13:08pitrousetstatus: open -> pending
resolution: fixed
messages: + msg135597

stage: patch review -> resolved
2011-05-09 15:10:42python-devsetnosy: + python-dev
messages: + msg135595
2011-05-07 16:53:50pitrousetfiles: + mpconn2.patch

messages: + msg135480
2011-04-03 20:51:02asksolsetmessages: + msg132881
2011-04-03 20:08:01alexsetnosy: + alex
2011-04-03 17:14:30s7v7nislandssetnosy: + s7v7nislands
2011-04-03 16:03:36pitrousetmessages: - msg132847
2011-04-03 16:03:30pitrousetfiles: - mpconn.patch
2011-04-03 16:03:24pitrousetmessages: + msg132848
2011-04-03 16:03:05pitrousetfiles: + pipebench.py

messages: + msg132847
2011-04-03 16:02:59pitrousetfiles: + mpconn.patch

messages: + msg132846
2011-04-03 12:41:06jnollersetmessages: + msg132835
2011-04-02 21:40:00pitroulinkissue9205 dependencies
2011-04-02 21:33:00pitroucreate