classification
Title: devguide suggests to use VS 2008 to build Python 2.7, but VS 2008 is no more supported?
Type: Stage:
Components: Build, Documentation, Windows Versions: Python 2.7
process
Status: open Resolution:
Dependencies: Superseder:
Assigned To: docs@python Nosy List: docs@python, eryksun, jkloth, paul.moore, steve.dower, tim.golden, vstinner, zach.ware
Priority: normal Keywords:

Created on 2017-05-12 09:32 by vstinner, last changed 2017-05-15 19:45 by steve.dower.

Messages (12)
msg293536 - (view) Author: STINNER Victor (vstinner) * (Python committer) Date: 2017-05-12 09:32
Steve Dower wrote this comment on my PR https://github.com/python/cpython/pull/1544 :

"Ah, I see. This is fine then - I have very little interest in doing much work to support the old build files (as they are explicitly unsupported)."

Copy of my reply:

"I have very little interest in doing much work to support the old build files (as they are explicitly unsupported)."

Wait wait wait... I fail to follow progress on the Windows build system, so I rely on the devguide which I expect to be up to date, and the guide says:

https://docs.python.org/devguide/setup.html#windows-compiling

"Python 2.7 uses Microsoft Visual Studio 2008, which is most easily obtained through an MSDN subscription."

I have a MSDN account, but after 30 min, I failed to get my MSDN number and I don't know my subscription is still valid or not (well, I guess that, it's expired). It took me 1 hour to find an old ISO of Visual Studio 2008, and not from microsoft.com :-(

I didn't want to install VS 2008, but I just followed the devguide.

... Do you mean that VS 2010 must now be used and the devguide is just outdated?


Moreover: we do have a buildbot using VS 2008. I'm now trying to fix all buildbots, so I had to fix issues on this VS 2008 buildbot too: http://bugs.python.org/issue30313

But I guess that it's ok to have a buildbot on a system not officially fully supported.
msg293550 - (view) Author: Jeremy Kloth (jkloth) * Date: 2017-05-12 14:29
A few things...

First to Victor's comments on the devguide.  The following sentence after the one quoted states:

"To use the build files in the PCbuild directory you will also need Visual Studio 2010"

In short, you need *both* VS2008 and VS2010 to compile the "default" way.

Secondly, to Steve's comment on the PC\VS9.0 build files.

Normally Steve would be correct in stating that the PC\VS* directories are not maintained, however, in this case they are supported.  It is explicitly stated as such in the devguide and PCbuild\readme.txt (https://github.com/python/cpython/blob/2.7/PCbuild/readme.txt).
msg293560 - (view) Author: Zachary Ware (zach.ware) * (Python committer) Date: 2017-05-12 16:40
Jeremy is correct.

To further clarify, *all* 2.7 Windows builders *are* using VS 2008's compiler toolchain, but they're using VS 2015's MSBuild with VS 2010's description of 2008's compiler toolchain to build the projects described in PCbuild/.  It's a mess, but it works and makes things much easier to customize.  The two specific VS9.0 builders are using VS 2008 directly via the PC/VS9.0 build files, but ideally the output from both methods should be exactly the same.  As Jeremy pointed out, the PC/VS9.0 files are still explicitly supported, and those two builders were set up to make sure they're not broken.

Anything older than PC/VS9.0 is explicitly *un*supported, and I would have no problem with just removing them (and any support for them in sysconfig).
msg293581 - (view) Author: STINNER Victor (vstinner) * (Python committer) Date: 2017-05-12 22:11
Anyway, I fixed some issues specific to VS9.0 recently, so things
should be a little bit better now :-)

For me, the last annoying issue is bpo-30313: re-run failed tests in
verbose mode. I will try to implement proposed solution.
msg293663 - (view) Author: Jeremy Kloth (jkloth) * Date: 2017-05-15 01:45
Now that my buildbot is back online, there is one unresolved issue pertaining to using VS9.0 only for building.  That is building without Perl.  It was a supported feature prior to backporting the MSBuild toolchain.  With recent OpenSSL releases, without Perl is no longer functional.

I have a changeset that restores this ability, but would like to know if that change was intentional, or just missed due to not having any buildbots covering VS9.0-only builds.
msg293667 - (view) Author: Zachary Ware (zach.ware) * (Python committer) Date: 2017-05-15 03:50
> I have a changeset that restores this ability, but would like to know if 
> that change was intentional, or just missed due to not having any buildbots 
> covering VS9.0-only builds.

It was intentional; it was more effort than I wanted to put in to get newer OpenSSL to work with both the new build scripts and the old method, so I figured installing Perl wasn't the worst thing in the world for those who wanted to use the VS9.0 files.  If your patch isn't too big or invasive, I'd be fine with applying it, though.
msg293686 - (view) Author: STINNER Victor (vstinner) * (Python committer) Date: 2017-05-15 08:40
"To further clarify, *all* 2.7 Windows builders *are* using VS 2008's compiler toolchain, but they're using VS 2015's MSBuild with VS 2010's description of 2008's compiler toolchain to build the projects described in PCbuild/."

Wait? Why not suggesting to use this recent VS 2015 if it's supported? It's a pain to try to install VS 2008 in 2017: it disappeared from microsoft.com, at least the Express edition. Moreover, it becomes more and more clear that VS 2008 has a worse support than the PCbuild directory.

So why not updating the devguide to suggest first to use the PCbuild directory using VS 2010 or newer? Are the latest VS version able to use PCbuild of Python 2.7?
msg293689 - (view) Author: STINNER Victor (vstinner) * (Python committer) Date: 2017-05-15 08:48
> Now that my buildbot is back online, there is one unresolved issue pertaining to using VS9.0 only for building.  That is building without Perl.  It was a supported feature prior to backporting the MSBuild toolchain.  With recent OpenSSL releases, without Perl is no longer functional.

I opened bpo-30368 to track this issue.
msg293707 - (view) Author: Jeremy Kloth (jkloth) * Date: 2017-05-15 10:42
> Wait? Why not suggesting to use this recent VS 2015 if it's
> supported?

In theory it is supported, yes.  However, it is not compatible with previous C runtimes.  Meaning things like memory allocation (malloc) and file pointers (FILE *) wouldn't play nice in extensions compiled against the 2008 toolchain (everything on PyPI).

> It's a pain to try to install VS 2008 in 2017: it disappeared from
> microsoft.com, at least the Express edition.

I believe that is why Microsoft released the Visual C++ Compiler for Python 2.7 (https://www.microsoft.com/en-us/download/details.aspx?id=44266) although, if memory serves, there is some issue with the 64-bit toolchain.

> Moreover, it becomes more and more clear that VS 2008 has a worse
> support than the PCbuild directory.

I guess that is why this bug report exists ;)  Seriously, though, that is why we recently added the VS9.0 builders.

> So why not updating the devguide to suggest first to use the
> PCbuild directory using VS 2010 or newer? Are the latest VS
> version able to use PCbuild of Python 2.7?

See above.  You can, but then your built Python is incompatible with the world.
msg293709 - (view) Author: STINNER Victor (vstinner) * (Python committer) Date: 2017-05-15 13:05
> I believe that is why Microsoft released the Visual C++ Compiler for Python 2.7 (https://www.microsoft.com/en-us/download/details.aspx?id=44266) although, if memory serves, there is some issue with the 64-bit toolchain.

Oh, I recall an announcement about that, but I don't see the link on
the devguide. It should be added.
msg293710 - (view) Author: Eryk Sun (eryksun) * Date: 2017-05-15 13:14
>> It's a pain to try to install VS 2008 in 2017: it disappeared from
>> microsoft.com, at least the Express edition.

> I believe that is why Microsoft released the Visual C++ Compiler for 
> Python 2.7 
> (https://www.microsoft.com/en-us/download/details.aspx?id=44266) 
> although, if memory serves, there is some issue with the 64-bit
> toolchain.

That release is intended for building extension modules. Is it possible to configure it for building CPython?
msg293734 - (view) Author: Steve Dower (steve.dower) * (Python committer) Date: 2017-05-15 19:45
> That release is intended for building extension modules. Is it possible to configure it for building CPython

It's intended for extension modules, but should be usable for building CPython itself if someone wants to take the time to update the Python 2.7 build system to use it. (Also, there's nothing wrong with the 64-bit toolset in this installer - that was the Windows SDK version of the compilers.)

The main reason you can't use it is because our build system is MSBuild based and the compiler package does not register with MSBuild (it doesn't provide a Platform Toolset). It would be possible to write one that would work, though I don't know exactly how to do it - I believe ICC and clang both provide the required files though, so it may be relatively easy to add the toolset files into our source repo and make it work.

There are likely some other reasons that the build will fail, but those can likely be worked around within the source repository.
History
Date User Action Args
2017-05-15 19:45:51steve.dowersetmessages: + msg293734
2017-05-15 13:14:11eryksunsetnosy: + eryksun
messages: + msg293710
2017-05-15 13:05:46vstinnersetmessages: + msg293709
2017-05-15 10:42:23jklothsetmessages: + msg293707
2017-05-15 08:48:02vstinnersetmessages: + msg293689
2017-05-15 08:40:58vstinnersetmessages: + msg293686
2017-05-15 03:50:26zach.waresetmessages: + msg293667
2017-05-15 01:45:21jklothsetmessages: + msg293663
2017-05-12 22:11:50vstinnersetmessages: + msg293581
2017-05-12 16:40:59zach.waresetmessages: + msg293560
2017-05-12 14:29:32jklothsetnosy: + jkloth
messages: + msg293550
2017-05-12 09:32:47vstinnercreate