classification
Title: dup_socket() on Windows should use WSA_FLAG_OVERLAPPED
Type: Stage: resolved
Components: Versions: Python 3.3
process
Status: closed Resolution: fixed
Dependencies: Superseder:
Assigned To: Nosy List: amaury.forgeotdarc, kristjan.jonsson, pitrou, python-dev, sbt
Priority: normal Keywords: patch

Created on 2012-03-14 12:59 by sbt, last changed 2012-04-01 20:15 by kristjan.jonsson. This issue is now closed.

Files
File name Uploaded Description Edit
socket_dup.patch sbt, 2012-03-14 12:59 review
Messages (8)
msg155748 - (view) Author: Richard Oudkerk (sbt) * (Python committer) Date: 2012-03-14 12:59
According to Microsoft's documentation sockets created using socket() have the
overlapped attribute, but sockets created with WSASocket() do not unless you 
pass the WSA_FLAG_OVERLAPPED flag.  The documentation for WSADuplicateSocket()
says

  If the source process uses the socket function to create the socket, the 
  destination process must pass the WSA_FLAG_OVERLAPPED flag to its WSASocket 
  function call.

This means that dup_socket() in socketmodule.c should use

    return WSASocket(FROM_PROTOCOL_INFO, FROM_PROTOCOL_INFO,
                     FROM_PROTOCOL_INFO, &info, 0, WSA_FLAG_OVERLAPPED);

instead of

    return WSASocket(FROM_PROTOCOL_INFO, FROM_PROTOCOL_INFO,
                     FROM_PROTOCOL_INFO, &info, 0, 0);

(On Windows, the new multiprocessing.connection.wait() function depends on
the overlapped attribute, although it is primarily intended for use with pipe 
connections not sockets.)

Patch attached.
msg155749 - (view) Author: Amaury Forgeot d'Arc (amaury.forgeotdarc) * (Python committer) Date: 2012-03-14 13:05
Which problem are you trying to solve?
Can this change be tested somehow?
msg155750 - (view) Author: Antoine Pitrou (pitrou) * (Python committer) Date: 2012-03-14 13:23
Are you sure this is desired? Nowhere can I think of a place in the stdlib where we use overlapped I/O on sockets.
msg155752 - (view) Author: Richard Oudkerk (sbt) * (Python committer) Date: 2012-03-14 14:43
pitrou wrote:
> Are you sure this is desired? Nowhere can I think of a place in the
> stdlib where we use overlapped I/O on sockets.

multiprocessing.connection.wait() does overlapped zero length reads on sockets.  It's documentation currently claims that it works with sockets.

Also it would seem strange if some sockets (created with socket()) have the overlapped attribute, but some others (created with WSASocket()) don't.

amaury.forgeotdarc wrote:
> Which problem are you trying to solve?

For one thing, the fact that socketmodule.c does not obey the word "must" in the quote from Microsoft's documentation.

> Can this change be tested somehow?

An additional test could be added to test_multiprocessing.TestWait.

Slightly surprisingly, in the testing I have done so far, using wait() with a duplicated socket seems to work without the patch.  However, I would be rather wary of just assuming that it works in all cases and on all versions of Windows.
msg157239 - (view) Author: Roundup Robot (python-dev) Date: 2012-03-31 23:25
New changeset 5be4d8fc9c44 by Antoine Pitrou in branch 'default':
Issue #14300: Under Windows, sockets created using socket.dup() now allow overlapped I/O.
http://hg.python.org/cpython/rev/5be4d8fc9c44
msg157240 - (view) Author: Antoine Pitrou (pitrou) * (Python committer) Date: 2012-03-31 23:26
Ok, I've committed the patch to 3.3 since it can be useful with the new wait() method. Thanks!
msg157326 - (view) Author: Kristján Valur Jónsson (kristjan.jonsson) * (Python committer) Date: 2012-04-01 19:57
I already included this fix in my "socket share" patch, see issue 14310.

I think this was a bug that should be checked in to all relevant branches.  The reason is this text from msdn documentation for WsaDuplicateSocket:
" Both the source process and the destination process should pass the same flags to their respective WSASocket function calls. If the source process uses the socket function to create the socket, the destination process _must_ [underline KVJ] pass the WSA_FLAG_OVERLAPPED flag to its WSASocket function call."

See http://msdn.microsoft.com/en-us/library/windows/desktop/ms741565(v=vs.85).aspx
msg157328 - (view) Author: Kristján Valur Jónsson (kristjan.jonsson) * (Python committer) Date: 2012-04-01 20:15
Also, see this: http://support.microsoft.com/kb/179942/EN-US
applies to windows 2000 only, as far as I can tell, though.  Don't know if we still support that.

I have scoured the docs, but found yet no reason to _not_ use this attribute.  I wonder why it is optional at all.
History
Date User Action Args
2012-04-01 20:15:32kristjan.jonssonsetmessages: + msg157328
2012-04-01 19:57:00kristjan.jonssonsetnosy: + kristjan.jonsson
messages: + msg157326
2012-03-31 23:26:04pitrousetstatus: open -> closed
resolution: fixed
messages: + msg157240

stage: resolved
2012-03-31 23:25:22python-devsetnosy: + python-dev
messages: + msg157239
2012-03-14 14:43:29sbtsetmessages: + msg155752
2012-03-14 13:23:48pitrousetnosy: + pitrou
messages: + msg155750
2012-03-14 13:05:54amaury.forgeotdarcsetnosy: + amaury.forgeotdarc
messages: + msg155749
2012-03-14 12:59:45sbtcreate