Author neologix
Recipients Albert.Strasheim, aljungberg, asksol, bquinlan, brian.curtin, gdb, gkcn, hongqn, jnoller, neologix, pitrou, vlasovskikh, vstinner
Date 2011-05-13.13:05:18
SpamBayes Score 1.50979e-12
Marked as misclassified No
Message-id <BANLkTikFRc7d=Cs0Q5-623e=iiTE_R9o3A@mail.gmail.com>
In-reply-to <1305284268.3561.7.camel@localhost.localdomain>
Content
> Not exactly. The select is done on the queue's pipe and on the workers'
> fds *at the same time*. Thus there's no race condition.

You're right, I missed this part, it's perfectly safe.

But I think there's a problem with the new implementation in Python.
Writes to a pipe are guaranteed to be atomic if you write less than
PIPE_BUF (4K on Linux, 512 by POSIX) at a time. Writes to a datagram
Unix domain socket are also atomic.
But Lib/multiprocessing/connection.py does:

    def _send_bytes(self, buf):
        # For wire compatibility with 3.2 and lower
        n = len(buf)
        self._send(struct.pack("=i", len(buf)))
        # The condition is necessary to avoid "broken pipe" errors
        # when sending a 0-length buffer if the other end closed the pipe.
        if n > 0:
            self._send(buf)

This is definitely not atomic. If two processes write objects of
different size at the same time, it can probably lead to trouble.
Also, Pipe(duplex=True) should probably return a SOCK_DGRAM Unix
socket for the same reason.
If I missed something here, I promise to shut up ;-)
History
Date User Action Args
2011-05-13 13:05:19neologixsetrecipients: + neologix, bquinlan, pitrou, vstinner, jnoller, hongqn, brian.curtin, asksol, vlasovskikh, gdb, Albert.Strasheim, aljungberg, gkcn
2011-05-13 13:05:18neologixlinkissue9205 messages
2011-05-13 13:05:18neologixcreate