multiprocessing's overlapped PipeConnection on Windows

[AntoinePitrou]

http://bugs.python.org/review/12328/diff/4322/15506
File Lib/multiprocessing/connection.py (right):

http://bugs.python.org/review/12328/diff/4322/15506#newcode309
Lib/multiprocessing/connection.py:309: ov, err = win32.ReadFile(self._handle,
128, overlapped=True)
Shouldn't it be bsize instead of 128? bsize looks otherwise unused.

http://bugs.python.org/review/12328/diff/4322/15506#newcode333
Lib/multiprocessing/connection.py:333: res = io.BytesIO()
One point of using BytesIO is to avoid repeated bytes concatenation, so you
might allocate it earlier and write() to it in _get_more_data.

http://bugs.python.org/review/12328/diff/4322/15506#newcode337
Lib/multiprocessing/connection.py:337: def _poll(self, timeout, maxsize=None):
Why does _poll take a maxsize?

http://bugs.python.org/review/12328/diff/4322/15506#newcode345
Lib/multiprocessing/connection.py:345: def _poll_and_not_empty_string(self):
Isn't the name of the method misleading, since it will return True if
_got_empty_message is True?
Also, shouldn't this be the same as calling _poll() with timeout=0?

http://bugs.python.org/review/12328/diff/4322/15506#newcode776
Lib/multiprocessing/connection.py:776: elif WAIT_OBJECT_0 <= res <=
WAIT_OBJECT_0 + len(L):
Shouldn't the second inequality sign be strict?
I.e. `... <= res < ...`.

http://bugs.python.org/review/12328/diff/4322/15506#newcode778
Lib/multiprocessing/connection.py:778: elif WAIT_ABANDONED_0 <= res <=
WAIT_ABANDONED_0 + len(L):
Same here.

http://bugs.python.org/review/12328/diff/4322/15506#newcode788
Lib/multiprocessing/connection.py:788: win32.ERROR_NETNAME_DELETED}
What does ERROR_NETNAME_DELETED mean? Does it occur with named pipes?
(and can 0 really occur?)

http://bugs.python.org/review/12328/diff/4322/15507
File Lib/multiprocessing/queues.py (right):

http://bugs.python.org/review/12328/diff/4322/15507#newcode103
Lib/multiprocessing/queues.py:103: # PipeConnection.poll() is not thread-safe. 
Since the
Actually, it looks like poll(0) should be thread-safe, if it's optimized to only
call PeekNamedPipe().

http://bugs.python.org/review/12328/diff/4322/15508
File Lib/test/test_multiprocessing.py (right):

http://bugs.python.org/review/12328/diff/4322/15508#newcode2486
Lib/test/test_multiprocessing.py:2486: class TestWait(unittest.TestCase):
It seems there are no tests for the timeout parameter, or for passing numeric
handles to wait().

http://bugs.python.org/review/12328/diff/4322/15509
File Modules/_multiprocessing/win32_functions.c (right):

http://bugs.python.org/review/12328/diff/4322/15509#newcode105
Modules/_multiprocessing/win32_functions.c:105: self->completed = 1;
Does ERROR_OPERATION_ABORTED still allow for some bytes to have been
transferred?

[sbt]

http://bugs.python.org/review/12328/diff/4322/15506
File Lib/multiprocessing/connection.py (right):

http://bugs.python.org/review/12328/diff/4322/15506#newcode309
Lib/multiprocessing/connection.py:309: ov, err = win32.ReadFile(self._handle,
128, overlapped=True)
On 2012/03/04 21:19:22, AntoinePitrou wrote:
> Shouldn't it be bsize instead of 128? bsize looks otherwise unused.
> 
Yes

http://bugs.python.org/review/12328/diff/4322/15506#newcode337
Lib/multiprocessing/connection.py:337: def _poll(self, timeout, maxsize=None):
On 2012/03/04 21:19:22, AntoinePitrou wrote:
> Why does _poll take a maxsize?
That is left over from an older version.

http://bugs.python.org/review/12328/diff/4322/15506#newcode345
Lib/multiprocessing/connection.py:345: def _poll_and_not_empty_string(self):
On 2012/03/04 21:19:22, AntoinePitrou wrote:
> Isn't the name of the method misleading, since it will return True if
> _got_empty_message is True?
> Also, shouldn't this be the same as calling _poll() with timeout=0?
You are right, this is no longer needed.

http://bugs.python.org/review/12328/diff/4322/15506#newcode776
Lib/multiprocessing/connection.py:776: elif WAIT_OBJECT_0 <= res <=
WAIT_OBJECT_0 + len(L):
On 2012/03/04 21:19:22, AntoinePitrou wrote:
> Shouldn't the second inequality sign be strict?
> I.e. `... <= res < ...`.
> 
Yes

http://bugs.python.org/review/12328/diff/4322/15506#newcode778
Lib/multiprocessing/connection.py:778: elif WAIT_ABANDONED_0 <= res <=
WAIT_ABANDONED_0 + len(L):
On 2012/03/04 21:19:22, AntoinePitrou wrote:
> Same here.
> 
Yes

http://bugs.python.org/review/12328/diff/4322/15506#newcode788
Lib/multiprocessing/connection.py:788: win32.ERROR_NETNAME_DELETED}
On 2012/03/04 21:19:22, AntoinePitrou wrote:
> What does ERROR_NETNAME_DELETED mean? Does it occur with named pipes?
> (and can 0 really occur?)
> 
It can occur when using overlapped reads on a socket (instead of
ERROR_BROKEN_PIPE).  And I don't think 0 should be in there.

http://bugs.python.org/review/12328/diff/4322/15507
File Lib/multiprocessing/queues.py (right):

http://bugs.python.org/review/12328/diff/4322/15507#newcode103
Lib/multiprocessing/queues.py:103: # PipeConnection.poll() is not thread-safe. 
Since the
On 2012/03/04 21:19:22, AntoinePitrou wrote:
> Actually, it looks like poll(0) should be thread-safe, if it's optimized to
only
> call PeekNamedPipe().
> 
In earlier versions of the patch, a successful poll would read a whole message
and not be atomic.  This is no longer necessary.

http://bugs.python.org/review/12328/diff/4322/15508
File Lib/test/test_multiprocessing.py (right):

http://bugs.python.org/review/12328/diff/4322/15508#newcode2486
Lib/test/test_multiprocessing.py:2486: class TestWait(unittest.TestCase):
On 2012/03/04 21:19:22, AntoinePitrou wrote:
> It seems there are no tests for the timeout parameter, or for passing numeric
> handles to wait().
> 
I will write some.

http://bugs.python.org/review/12328/diff/4322/15509
File Modules/_multiprocessing/win32_functions.c (right):

http://bugs.python.org/review/12328/diff/4322/15509#newcode105
Modules/_multiprocessing/win32_functions.c:105: self->completed = 1;
On 2012/03/04 21:19:22, AntoinePitrou wrote:
> Does ERROR_OPERATION_ABORTED still allow for some bytes to have been
> transferred?
> 
I don't think so.  But the only cases we really care about are (1) zero length
reads where I am sure it does happen, and (2) cases where the read is
interrupted by SIGINT where potential corruption is just as much of a problem on
Unix.