This issue tracker has been migrated to GitHub, and is currently read-only.
For more information, see the GitHub FAQs in the Python's Developer Guide.

Author ryles
Recipients jnoller, ryles
Date 2009-05-19.05:19:32
SpamBayes Score 1.9575161e-09
Marked as misclassified No
Message-id <1242710426.4.0.944213497098.issue6056@psf.upfronthosting.co.za>
In-reply-to
Content
Yeah, storing pickled queues in the file system makes for some easy IPC
:) It wasn't a very original idea, I took the pickling comments in the
documentation at face value:
http://docs.python.org/library/multiprocessing.html#proxy-objects

So, from what I can tell this issue is related to the mixing of standard
python socket I/O with multiprocessing socket I/O, with state not being
carried from the former to the latter.

In multiprocessing/connection.py, SocketClient() creates a familiar
python socket object which, when a default timeout has been set in the
module, will be made non-blocking. In addition, the timeout is
remembered in the socket object, and when calling socket.recv(), the
function internal_select() will use this to perform the expected poll()
or select().

However, after a connection is established, SocketClient() will not use
python's socket implementation any further, and instead pass its
low-level socket descriptor to a multiprocessing Connection object. This
object has its own specialized socket I/O implementation, which is not
at all aware of the timeout previously associated with the socket. As a
result no select/poll occurs and, due to the socket's non-blocking
status, recv() may return EAGAIN immediately. I suspect this is what's
happening.

There might be a number of ways to make SocketClient() more timeout
friendly, but possibility could be to simply check if the python socket
has a timeout associated, and if so, use connection.poll() in addition
to connection.recv(). There may be other places in the code where
similar changes would occur.

You obviously have more experience with this code base so I'll be
curious to see the outcome.
History
Date User Action Args
2009-05-19 05:20:38rylessetrecipients: + ryles, jnoller
2009-05-19 05:20:26rylessetmessageid: <1242710426.4.0.944213497098.issue6056@psf.upfronthosting.co.za>
2009-05-19 05:20:17ryleslinkissue6056 messages
2009-05-19 05:19:52rylescreate