classification
Title: _ctypes and other modules not built with msbuild on vs2010 solution
Type: Stage:
Components: Build Versions: Python 3.3
process
Status: closed Resolution: fixed
Dependencies: Superseder:
Assigned To: brian.curtin Nosy List: brian.curtin, jaraco, kristjan.jonsson, pitrou, python-dev
Priority: normal Keywords:

Created on 2012-05-16 00:10 by jaraco, last changed 2012-05-24 12:28 by jaraco. This issue is now closed.

Messages (9)
msg160780 - (view) Author: Jason R. Coombs (jaraco) * (Python committer) Date: 2012-05-16 00:10
It appears with the latest changes (cdcc816dea85), ctypes builds as _ctypes_test.pyd, not _ctypes.pyd. This causes 'import ctypes' to fail with an ImportError and thus causes tests to fail that depend on test.support (which imports ctypes).

I first noticed this issue in features/pep-420, but the same behavior is present in the main repository. I cannot test it because Python fails to start altogether, but I see _ctypes_test.pyd in the pcbuild/amd64 directory.
msg160787 - (view) Author: Jason R. Coombs (jaraco) * (Python committer) Date: 2012-05-16 00:51
I see now that _ctypes is in fact in the solution, and if I open the solution in visual studio and build it manually, it gets built. Even if I just build the solution in visual studio, it gets built.

However, if I run the following command, _ctypes doesn't get built:

msbuild pcbuild.sln /p:Configuration=Release /p:Platform=x64

The output log doesn't mention "ctypes.", so it appears as if it's not attempting to build it at all.

This would work for me a week or two ago, prior to the VS2010 conversion (though I would convert the project before building), so this behavior feels a bit like a regression.
msg160795 - (view) Author: Jason R. Coombs (jaraco) * (Python committer) Date: 2012-05-16 01:57
I've discovered a few things:

- _ctypes depends on pythoncore.
- other projects that depend on pythoncore are not built by msbuild.
- If I remove the dependency of _ctypes on pythoncore and then do the build with msbuild, _ctypes is built and I can import ctypes.
- If instead I explicitly specify _ctypes as a target, it gets built (and along with a regular build, I can import ctypes).

I see why that ctypes does depend on pythoncore, because it needs python33.lib to link.

In trying to discover how to have msbuild attempt to build the entire solution, I ran across [this article](http://blogs.msdn.com/b/visualstudio/archive/2010/12/21/incorrect-solution-build-ordering-when-using-msbuild-exe.aspx) which says that best practice is not to declare project dependencies in a solution file at all but instead declare project dependencies in the project files themselves (so _ctypes should <Include> pythoncore, for example).

I plan to look into this issue more later.
msg160798 - (view) Author: Brian Curtin (brian.curtin) * (Python committer) Date: 2012-05-16 05:00
Can you try http://bugs.python.org/file25583/pcbuildpatch.patch from #13210?
msg161202 - (view) Author: Kristján Valur Jónsson (kristjan.jonsson) * (Python committer) Date: 2012-05-20 11:09
I'm able to reproduce this problem with the latest changes to PCBuild.
Are you saying that there are other projects like _ctypes that are not built?

I'm not able to see any different between _ctypes_test and _ctypes on the .vcxsproj level that would explain this, or in the sln.

Perhaps there is another bug in msbuild?
msg161203 - (view) Author: Kristján Valur Jónsson (kristjan.jonsson) * (Python committer) Date: 2012-05-20 11:12
Hm, I see that this applies to _decimal, _testbuffer and xxlimited too.  When msbuild is invoked with MSBUILDEMITSOLUTION=1, it leaves a .metaproj file in place for those projects.
msg161204 - (view) Author: Kristján Valur Jónsson (kristjan.jonsson) * (Python committer) Date: 2012-05-20 11:24
Found the issue.  Manual dependencies in the .sln file had to be removed.
msg161205 - (view) Author: Roundup Robot (python-dev) (Python triager) Date: 2012-05-20 11:27
New changeset 2e96629c6dab by Kristján Valur Jónsson in branch 'default':
Issue 14821:
http://hg.python.org/cpython/rev/2e96629c6dab
msg161501 - (view) Author: Jason R. Coombs (jaraco) * (Python committer) Date: 2012-05-24 12:28
Excellent! The latest tip now builds nicely using msbuild.
History
Date User Action Args
2012-05-24 12:28:21jaracosetstatus: open -> closed
resolution: fixed
messages: + msg161501
2012-05-20 11:27:16python-devsetnosy: + python-dev
messages: + msg161205
2012-05-20 11:24:06kristjan.jonssonsetmessages: + msg161204
2012-05-20 11:12:40kristjan.jonssonsetmessages: + msg161203
2012-05-20 11:09:21kristjan.jonssonsetnosy: + kristjan.jonsson
messages: + msg161202
2012-05-20 01:59:45jaracosettitle: _ctypes_test fails to build from the command line with VS 2010 -> _ctypes and other modules not built with msbuild on vs2010 solution
2012-05-19 21:37:55loewissettitle: Ctypes extension module builds as _ctypes_test.pyd -> _ctypes_test fails to build from the command line with VS 2010
2012-05-16 05:00:20brian.curtinsetmessages: + msg160798
2012-05-16 01:57:06jaracosetmessages: + msg160795
2012-05-16 00:51:07jaracosetmessages: + msg160787
2012-05-16 00:17:33ezio.melottisetnosy: + pitrou
2012-05-16 00:10:11jaracocreate