classification
Title: test_socket fails using Visual Studio 2010
Type: behavior Stage:
Components: Build, Library (Lib) Versions: Python 3.2, Python 3.1, Python 2.7
process
Status: closed Resolution: fixed
Dependencies: Superseder: Support Visual Studio 2010
View: 13210
Assigned To: Nosy List: Kotan, Simon, amaury.forgeotdarc, brian.curtin, kristjan.jonsson, loewis, pitrou, sable, tim.golden
Priority: normal Keywords: patch

Created on 2010-11-20 14:24 by Kotan, last changed 2012-06-11 13:15 by kristjan.jonsson. This issue is now closed.

Files
File name Uploaded Description Edit
test_socket.log Kotan, 2010-11-20 14:24 regrtest output
pcbuild_vs2010.patch Kotan, 2010-11-20 17:25 patch for project and solution files for VS 2010 express edition
issue10469.patch Kotan, 2010-11-20 21:33 switch preference of defines to prefer WSA* alternatives review
cpython_75849_to_75851.diff kristjan.jonsson, 2012-03-21 16:19 WSA error patch from CCP review
Messages (18)
msg121651 - (view) Author: Daniel Albeseder (Kotan) Date: 2010-11-20 14:24
Using WinXP I compiled python 3.2 from the current sources using Visual C++ 2010 Express.

Running

rt -v test_socket

resulted in the attached output.

The tests testSmallReadNonBlocking and testWriteNonBlocking have errors, and an assertion fails for test_connect and test_create_connections.
msg121652 - (view) Author: Daniel Albeseder (Kotan) Date: 2010-11-20 14:28
current revision = svn revision 86553
msg121659 - (view) Author: Antoine Pitrou (pitrou) * (Python committer) Date: 2010-11-20 14:59
The core problem seems to be that the errno module has mismatching values on your Python build (for ECONNREFUSED and EWOULDBLOCK, at least, which explains all 4 failures). On a Windows 7 VM here:

>>> errno.ECONNREFUSED
10061
>>> errno.EWOULDBLOCK
10035
msg121688 - (view) Author: Daniel Albeseder (Kotan) Date: 2010-11-20 17:25
As this was requested on IRC, I put the Visual C++ 2010 Express solution and project files here as well. The patch "pcbuild_vs2010.patch" should be applied to the PCbuild directory.
msg121692 - (view) Author: Brian Curtin (brian.curtin) * (Python committer) Date: 2010-11-20 17:35
Daniel: If you need VS2008, you can get it here: #2008-Visual-CPP">http://www.microsoft.com/express/Downloads/#2008-Visual-CPP
msg121738 - (view) Author: Martin v. Löwis (loewis) * (Python committer) Date: 2010-11-20 19:40
Kotan: please perform the following procedure:

1. Open pcbuild.sln
2. In the pythoncore project, open errnomodule.c
3. find the reference to EWOULDBLOCK and WSAEWOULDBLOCK
4. for each one, "go to definition"
5. report here what header files Visual Studio thinks they are defined in, and what values they have.
msg121741 - (view) Author: Daniel Albeseder (Kotan) Date: 2010-11-20 19:50
#define EWOULDBLOCK     140
#define WSAEWOULDBLOCK                   10035L

The editor suggests that EWOULDBLOCK is already defined, therefore its code is used, i.e. 140.

#define ECONNREFUSED    107
#define WSAECONNREFUSED                  10061L

The E* are defined in errno.h and the WSA* in WinError.h. It seems that in my case the POSIX compliant headers are used.
msg121748 - (view) Author: Martin v. Löwis (loewis) * (Python committer) Date: 2010-11-20 20:07
OK, so what's the path to errno.h? In the errno.h included in my copy of VS 2010, the errno values only go up to 42 (80 if you count STRUNCATE), and EWOULDBLOCK is not defined in this header file.
msg121753 - (view) Author: Daniel Albeseder (Kotan) Date: 2010-11-20 20:14
c:\Program Files\Microsoft Visual Studio 10.0\VC\include\errno.h

The content of my errno.h:

...
#define EILSEQ          42
#define STRUNCATE       80
#endif
#endif

/* Support EDEADLOCK for compatibility with older MS-C versions */
#define EDEADLOCK       EDEADLK

/* POSIX SUPPLEMENT */
#define EADDRINUSE      100
#define EADDRNOTAVAIL   101
...
msg121762 - (view) Author: Martin v. Löwis (loewis) * (Python committer) Date: 2010-11-20 20:34
Oops, I was confused. My copy is VS 2008, of course. So that's another incompatible change in VS 2010. Tcl, Ruby, and curl are ahead of us, here:

http://redmine.ruby-lang.org/issues/show/3092
http://sourceforge.net/tracker/index.php?func=detail&aid=3019634&group_id=10894&atid=310894
http://www.mail-archive.com/curl-library@cool.haxx.se/msg05164.html

I think we can work around that by switching the order of preference: use WSA constants if available, and errno.h values otherwise.

Kotan: can you please search the CRT sources whether these "POSIX" values are used anywhere?
msg121767 - (view) Author: Daniel Albeseder (Kotan) Date: 2010-11-20 20:51
Only in the "system_error" file. (http://msdn.microsoft.com/en-us/library/ee372194.aspx) Seem to be a C++0X standard header file.

http://msdn.microsoft.com/en-us/library/5814770t.aspx

talks about the Posix codes only for compatibility reasons.
msg121770 - (view) Author: Martin v. Löwis (loewis) * (Python committer) Date: 2010-11-20 20:55
> Only in the "system_error" file. (http://msdn.microsoft.com/en-us/library/ee372194.aspx) 
> Seem to be a C++0X standard header file.

(I assume you have actually scanned the crt\src folder as well)

Then it's indeed safe to ignore.

> http://msdn.microsoft.com/en-us/library/5814770t.aspx
> 
> talks about the Posix codes only for compatibility reasons.

Unfortunately, they managed to break compatibility with that change,
instead of introducing it.
msg121784 - (view) Author: Daniel Albeseder (Kotan) Date: 2010-11-20 21:33
Ok I did switched the preference for all "new" defines inside VC++ errno.h to use the WSA* alternatives if available.

Now test_socket passes for me, and the test_asyncore which did block before, now passes without a problem.
msg141149 - (view) Author: Simon Buchan (Simon) Date: 2011-07-26 12:15
Confirming this patch fixes the test_aynsc* tests in my VS10 build. Shouldn't it swap all WSA* defines to protect against this in the future, though? Alternatively, should the check for WSA* codes existing be in Lib\asyncore.py?
msg147889 - (view) Author: Sébastien Sablé (sable) Date: 2011-11-18 16:32
I can also confirm that this patch corrects the problem with test_asyncore for Python 2.7 built with VS2010.
msg147891 - (view) Author: Brian Curtin (brian.curtin) * (Python committer) Date: 2011-11-18 16:36
FYI: this would likely be handled through #13210. I have a conversion sandbox started at http://hg.python.org/sandbox/vs2010port/ and am working through fixing test failures after the initial conversion.
msg156495 - (view) Author: Kristján Valur Jónsson (kristjan.jonsson) * (Python committer) Date: 2012-03-21 16:19
Added a patch as used by CCP in production.  Covers more WSA cases.
msg162614 - (view) Author: Kristján Valur Jónsson (kristjan.jonsson) * (Python committer) Date: 2012-06-11 13:15
This has been fixed with the proper 2010 support
History
Date User Action Args
2012-06-11 13:15:18kristjan.jonssonsetstatus: open -> closed
superseder: Support Visual Studio 2010
resolution: fixed
messages: + msg162614
2012-03-21 16:19:25kristjan.jonssonsetfiles: + cpython_75849_to_75851.diff
nosy: + kristjan.jonsson
messages: + msg156495

2011-11-18 16:36:46brian.curtinsetmessages: + msg147891
2011-11-18 16:32:15sablesetnosy: + sable
messages: + msg147889
2011-07-26 12:15:44Simonsetnosy: + Simon
messages: + msg141149
2010-11-20 22:23:20Kotansettitle: test_socket fails -> test_socket fails using Visual Studio 2010
2010-11-20 21:33:15Kotansetfiles: + issue10469.patch

messages: + msg121784
2010-11-20 20:55:37loewissetmessages: + msg121770
2010-11-20 20:51:46Kotansetmessages: + msg121767
2010-11-20 20:34:42loewissetmessages: + msg121762
2010-11-20 20:14:53Kotansetmessages: + msg121753
2010-11-20 20:07:11loewissetmessages: + msg121748
2010-11-20 19:50:21Kotansetmessages: + msg121741
2010-11-20 19:40:58loewissetmessages: + msg121738
2010-11-20 17:35:36brian.curtinsetmessages: + msg121692
2010-11-20 17:25:57Kotansetfiles: + pcbuild_vs2010.patch
keywords: + patch
messages: + msg121688
2010-11-20 15:21:15pitrousetnosy: + amaury.forgeotdarc
2010-11-20 14:59:20pitrousetversions: + Python 3.1, Python 2.7
nosy: + pitrou

messages: + msg121659

components: + Build
type: behavior
2010-11-20 14:52:44pitrousetnosy: + loewis, tim.golden, brian.curtin
2010-11-20 14:28:45Kotansetmessages: + msg121652
2010-11-20 14:24:40Kotancreate