The buildbot scripts do not work for the 64-bit targets.

Firstly, the "/p:UseEnv=True" parameter to msbuild causes the 32-bit only projects (make_buildinfo and make_versioninfo) to fail due to architecture mismatch.  The scripts now unconditionally build those projects using the 32-bit command-line tools.

The make_versioninfo project wasn't completely converted to be 32-bit only (issue9981); it would work correctly only from the IDE.  This fixes that and removes the Debug configuration as that was already #ifdef'd in the generated .h file.

There's some unrelated stuff in the patch that reverts commits in
mpdecimal.c and multiprocessing. Maybe a Rietveld issue.

The unrelated changes do seem to be an issue with the diff generated by Rietveld

If I understand, the proposed changes are the following:
https://bitbucket.org/jkloth/cpython-buildbot-amd64/changeset/2a20cee18add
?

I'm not a Windows specialist, but they do seem a bit weird:
- the pcbuild.sln changes remove "Debug|x64.Build" and replaces "Debug" with "Release"
- build-common.bat doesn't allow to distinguish between x86 and x64
- build.bat does not call build-common.bat

On Mon, Jun 18, 2012 at 3:27 PM, Antoine Pitrou <report@bugs.python.org> wrote:
> If I understand, the proposed changes are the following:
> https://bitbucket.org/jkloth/cpython-buildbot-amd64/changeset/2a20cee18add

That seems to be correct, yes.

> I'm not a Windows specialist, but they do seem a bit weird:
> - the pcbuild.sln changes remove "Debug|x64.Build" and replaces "Debug" with "Release"

That change is related to the make_versioninfo project which doesn't
use the Debug build (like make_buildinfo)

> - build-common.bat doesn't allow to distinguish between x86 and x64

Correct as make_buildinfo and make_versioninfo are only ever built as
32-bit (see issue9981)

> - build.bat does not call build-common.bat

Oops, an oversight on my part.  It didn't show up in testing as the
solution already is built in 32-bit mode.  The call should in the same
location as in build-amd64.bat

Now that my buildbot is up and building (and failing at that) could these changes be committed?  It would get the buildbot to at least start testing changes to Python proper.

> Now that my buildbot is up and building (and failing at that) could
> these changes be committed?

Well, it would be nice if you could post an updated patch after the comments above. (I'm not saying I'm gonna commit, since I'm not a Windows expert, but it will certainly help other core developers review it)

I'd like Martin to have a look at this.

I think this really needs to be broken further down than "the AMD64 buildbot doesn't work". For each presumed failure, there should be one issue, and perhaps a proposed patch.

The patch, as it stands, is clearly incorrect (as has been reviewed and acknowledged).

So unless Jeremy can provide an updated patch quickly, I rather suggest to close this issue, and ask for new issues to be created for the individual problems.

As for the changes to project files and the solution file: none of these changes should be necessary at all, unless it's actually the case that the AMD64-Debug build doesn't work (i.e. neither in the buildbot, nor from the solution file).

> The patch, as it stands, is clearly incorrect (as has been reviewed and acknowledged).

The most recent patch is correct.

> As for the changes to project files and the solution file: none of these changes should be necessary at all, unless it's actually the case that the AMD64-Debug build doesn't work (i.e. neither in the buildbot, nor from the solution file).

The debug build doesn't even *build*!  That's the issue.

> As for the changes to project files and the solution file: none of these changes should be necessary at all, unless it's actually the case that the AMD64-Debug build doesn't work (i.e. neither in the buildbot, nor from the solution file).

As to the debug build not working, has anyone even looked at the
64-bit Windows buildbot? As the following link demonstrates, building
is clearly failing:

http://www.python.org/dev/buildbot/all/builders/AMD64%20Windows7%20SP1%203.x/builds/74/steps/compile/logs/stdio

> The most recent patch is correct.

See my review then; I think it has issues.

> The debug build doesn't even *build*!  That's the issue.

It builds fine for me - I just tried.

Martin v. L??wis <report@bugs.python.org> wrote:
> > The debug build doesn't even *build*!  That's the issue.
> 
> It builds fine for me - I just tried.

What command line do you use? I tried:

C:\Users\stefan\pydev\cpython>.\Tools\buildbot\build-amd64.bat

Error: project file 'pcbuild.sln' was not found or not a valid project file.
Microsoft (R) Visual C++ Project Builder - Command Line Version 9.00.21022 
Copyright (C) Microsoft Corporation. All rights reserved.

>> > The debug build doesn't even *build*!  That's the issue.
>>
>> It builds fine for me - I just tried.
>
> What command line do you use?

Not the command line - I built from the IDE. My claim is if that works,
no changes to the solution and project files should be necessary.

> C:\Users\stefan\pydev\cpython>.\Tools\buildbot\build-amd64.bat
>
> Error: project file 'pcbuild.sln' was not found or not a valid project file.

I can believe that there are issues with the batch files. That's why I propose
to look at one issue at a time (but it's not my priority at the moment).

> I can believe that there are issues with the batch files. That's why I propose
> to look at one issue at a time (but it's not my priority at the moment).

That would mean that you do not care that the buildbots are failing.
That is exactly the reason that this bug report has been opened.

Of course these failures have to be fixed.  But what Martin is saying is that for him (and probably other devs working on Windows) the build works, so the change must be done in the scripts only running on the buildbots.

As for priority, we're two days away from 3.3b1 and feature freeze, so you can imagine that there are quite a lot of things on Martin's table.

I seem to be getting a functioning amd64 debug build by applying
the small useenv.diff and then running:

Tools\buildbot\build-amd64.bat

> This change eliminates the need for HOST_PYTHON.  Why is it required
> in the first place?  It is not documented as being required to build
> Python.

It helps to build 64-bit binaries on a 32-bit Windows installation.
Normally, Python uses the python.exe it just built, but this fails if
the binary is a 64-bit binary, but the host is 32-bits.

It would be best if HOST_PYTHON was used if set, but it otherwise
falls back to using the Python just built.

I'm fine with dropping HOST_PYTHON for 3.4, but not for 3.3.

> When building in the VS IDE, the proper compilers are chosen.
> However, when building via the buildbot scripts, the compilers are
> forced to the 64-bit only versions and thus cause link errors.  See
> the buildbot status for the exact error messages.

I see the errors, but I believe the solution is incorrect. It must
be possible to build from the command line exactly in the same way
as from the IDE, and it *was* possible to do so with VS 2008. We
just need to find out how to do it with VS 2010.

> This file is a redundant with python_nt.h. It was removed for simplicity.

That is out of scope for this issue, then. If the file annoys you much,
create a separate issue.

> I do not know what you reviewed, but the diff I generated does not
> have those changes.

Interesting. Now Rietveld doesn't show them anymore, either, in the place
where I left the comment.

Stefan: your patch looks good, please apply.

New changeset 979567d33376 by Stefan Krah in branch 'default':
Issue #15102: Allow platform-specific settings for the current project
http://hg.python.org/cpython/rev/979567d33376

New changeset a9d4cf7d15b9 by Martin v. Löwis in branch 'default':
Issue #15102: Use HOST_PYTHON only if it is set.
http://hg.python.org/cpython/rev/a9d4cf7d15b9

I've fixed some paths in external-amd64.bat. Uploading a diff
instead of committing right away so as not to interfere with
the steadily incoming commits.

Stefan, don't fear interfering. Merging is easy.

New changeset c56783fe2e3b by Stefan Krah in branch 'default':
Issue #15102: Fix paths in external-amd64.bat.
http://hg.python.org/cpython/rev/c56783fe2e3b

Antoine Pitrou <report@bugs.python.org> wrote:
> Stefan, don't fear interfering. Merging is easy.

Good to know, I loathe merging. :)

There's still a problem when building ssl:

Project "C:\Users\Buildbot\buildbot.python.org\3.x.kloth-win64\build\PCbuild\pcbuild.sln" (1) is building "C:\Users\Buildbot\buildbot.python.org\3.x.kloth-win64\build\PCbuild\_ssl.vcxproj" (18) on node 1 (default targets).
Project "C:\Users\Buildbot\buildbot.python.org\3.x.kloth-win64\build\PCbuild\_ssl.vcxproj" (18) is building "C:\Users\Buildbot\buildbot.python.org\3.x.kloth-win64\build\PCbuild\ssl.vcxproj" (19) on node 1 (default targets).
Build:
  cd "C:\Users\Buildbot\buildbot.python.org\3.x.kloth-win64\build\PCbuild\"
  "C:\Users\Buildbot\buildbot.python.org\3.x.kloth-win64\build\PCbuild\python_d.exe" build_ssl.py Release x64 -a
  
  '"C:\Users\Buildbot\buildbot.python.org\3.x.kloth-win64\build\PCbuild\python_d.exe"' is not recognized as an internal or external command,
  operable program or batch file.


(from http://www.python.org/dev/buildbot/all/builders/AMD64%20Windows7%20SP1%203.x/builds/90/steps/compile/logs/stdio )

New changeset 9c71d7f057b3 by Martin v. Löwis in branch 'default':
Issue #15102: find python.exe in OutDir, not SolutionDir.
http://hg.python.org/cpython/rev/9c71d7f057b3

All Windows buildbots are failing now:

  C:\Program Files\MSBuild\Microsoft.Cpp\v4.0\Microsoft.MakeFile.Targets(38,5): error MSB3073: The command "cd "d:\cygwin\home\db3l\buildarea\3.x.bolen-windows\build\PCbuild\" [d:\cygwin\home\db3l\buildarea\3.x.bolen-windows\build\PCbuild\ssl.vcxproj]
C:\Program Files\MSBuild\Microsoft.Cpp\v4.0\Microsoft.MakeFile.Targets(38,5): error MSB3073: "d:\cygwin\home\db3l\buildarea\3.x.bolen-windows\build\PCbuild\Debug\python_d.exe" build_ssl.py Release Win32 -a [d:\cygwin\home\db3l\buildarea\3.x.bolen-windows\build\PCbuild\ssl.vcxproj]
C:\Program Files\MSBuild\Microsoft.Cpp\v4.0\Microsoft.MakeFile.Targets(38,5): error MSB3073: " exited with code 3. [d:\cygwin\home\db3l\buildarea\3.x.bolen-windows\build\PCbuild\ssl.vcxproj]



Perhaps it would be possible to set HOST_PYTHON in build-amd64.bat
as a temporary measure until a more elegant solution is found?

New changeset fc84ac3e8dfc by Stefan Krah in branch 'default':
Issue #15102: Try setting HOST_PYTHON in build-amd64.bat as a temporary
http://hg.python.org/cpython/rev/fc84ac3e8dfc

I'm trying out my suggestion on the 64-bit bot. Will revert if it
doesn't work out.

Well, the path to python_d.exe looks right now, but Perl and nasm
aren't found:


"C:\Users\Buildbot\buildbot.python.org\3.x.kloth-win64\build\PCbuild\amd64\python_d.exe" build_ssl.py Release x64 -a
  
  Found a working perl at 'C:\perl\bin\perl.exe'
  'nasm' is not recognized as an internal or external command,
  operable program or batch file.
  nasm assembler has failed.



Jeremy, did you install these on your machine?

> I'm trying out my suggestion on the 64-bit bot. Will revert if it
> doesn't work out.

Even if it does work out it should be reverted. As HOST_PYTHON is not
set, PythonExe should be computed correctly. So we really need to find
out why it isn't.

On 24.06.2012 13:22, Stefan Krah wrote:
> 
> Stefan Krah <stefan-usenet@bytereef.org> added the comment:
> 
> Well, the path to python_d.exe looks right now, but Perl and nasm
> aren't found:

perl shouldn't be necessary. Not sure why nasm is.

Martin v. L??wis <report@bugs.python.org> wrote:
> Even if it does work out it should be reverted. As HOST_PYTHON is not
> set, PythonExe should be computed correctly. So we really need to find
> out why it isn't.

Yes, of course. It's meant as a temporary measure to work out things like
the missing nasm and hopefully get a functioning bot before the beta.

> Well, the path to python_d.exe looks right now, but Perl and nasm
> aren't found:
> 
> 
> "C:\Users\Buildbot\buildbot.python.org\3.x.kloth-win64\build\PCbuild\amd64\python_d.exe" build_ssl.py Release x64 -a
>   
>   Found a working perl at 'C:\perl\bin\perl.exe'

Hmm, obviously Perl *is* found. That leaves nasm.

> Hmm, obviously Perl *is* found. That leaves nasm.

nasm.exe was installed for myself (hence testing worked).  I've now
made it available to the buildbot account as well.

Note that I have Perl installed but *not* on my PATH (on purpose).
The patch supplied in issue15052 will fix the issue brought out by
that circumstance.

Jeremy Kloth <report@bugs.python.org> wrote:
> nasm.exe was installed for myself (hence testing worked).  I've now
> made it available to the buildbot account as well.
> 
> Note that I have Perl installed but *not* on my PATH (on purpose).
> The patch supplied in issue15052 will fix the issue brought out by
> that circumstance.

Thanks. The Perl situation looks OK since the buildbot has proceeded to
the testing stage in the last build.

Jeremy, your python-dev suggestion to move x64.props before pyproject.props
does not work for me. For instance, in python3dll.vcxproj:

 C:\Program Files (x86)\MSBuild\Microsoft.Cpp\v4.0\Microsoft.MakeFile.Targets(38,5): error MSB3073: The comma 
nd "cd C:\Users\stefan\pydev\cpython\PCbuild\\..\PC [C:\Users\stefan\pydev\cpython\PCbuild\python3dll.vcxproj] 
C:\Program Files (x86)\MSBuild\Microsoft.Cpp\v4.0\Microsoft.MakeFile.Targets(38,5): error MSB3073: nmake /f py 
thon3.mak MACHINE=x64 OutDir=C:\Users\stefan\pydev\cpython\PCbuild\x64-pgo\" exited with code 9009. [C:\Users\ 
stefan\pydev\cpython\PCbuild\python3dll.vcxproj]

Ouch. The test failures on the buildbot now seem to point to problems with OpenSSL's SHA1 implementation.

Jeremy, perhaps you can try to run the OpenSSL self-tests?

outdir.diff contains a solution that doesn't rely on HOST_PYTHON.
Works for both x64 and Win32.

Note that I've edited the files using Vim. -- I'm not sure if one can
even select Condition="'$(Platform)'=='Win32'" using Visual Studio.

Stefan: I think that's acceptable. Perhaps someone will come up with a better solution; if this works, it's good enough for me.

New changeset f12a2f2c93e5 by Stefan Krah in branch 'default':
Issue #15102: Fix paths of python executables.
http://hg.python.org/cpython/rev/f12a2f2c93e5

Note that there also appears to be a compile error on the 3.2 and 2.7 branches:

http://buildbot.python.org/all/builders/AMD64%20Windows7%20SP1%203.2
http://buildbot.python.org/all/builders/AMD64%20Windows7%20SP1%202.7

Build started: Project: ssl, Configuration: Debug|x64
Performing Makefile project actions
'""' is not recognized as an internal or external command,
operable program or batch file.
Project : error PRJ0019: A tool returned an error code from "Performing Makefile project actions"
Project : warning PRJ0018 : The following environment variables were not found:
$(HOST_PYTHON)
Build log was saved at "file://C:\Users\Buildbot\buildbot.python.org\3.2.kloth-win64\build\PCbuild\x64-temp-Debug\ssl\BuildLog.htm"
ssl - 1 error(s), 0 warning(s)

> Note that there also appears to be a compile error on the 3.2 and 2.7 branches:

Can we please, pretty pretty please, deal with one issue at a time?

If you want to keep this issue open until everything works, you will
probably see a dozen more commits, and wait several more months.

As far as I can see, the 3.3 buildbot scripts are fixed: After af1d98b7ca80
I can run Tools/buildbot/build.bat and Tools/buildbot/build-amd64.bat
successfully in any order.

I've run the tests, too, and I'm not getting the SHA1 failures.

As requested, I've opened separate issues for the 2.7/3.2 scripts
and one for the SHA1 failures on the bot.

Hmm, the buildbot fails compiling again after af1d98b7ca801049803d83a85bb8558f6e59f2a1:

http://buildbot.python.org/all/builders/AMD64%20Windows7%20SP1%203.x/builds/139

Antoine Pitrou <report@bugs.python.org> wrote:
> Hmm, the buildbot fails compiling again after af1d98b7ca801049803d83a85bb8558f6e59f2a1:
> 
> http://buildbot.python.org/all/builders/AMD64%20Windows7%20SP1%203.x/builds/139

I think it's unrelated (probably ssl maintenance going on):

cd "C:\Users\Buildbot\buildbot.python.org\3.x.kloth-win64\build\PCbuild\"
  "C:\Users\Buildbot\buildbot.python.org\3.x.kloth-win64\build\PCbuild\amd64\python_d.exe" build_ssl.py Release x64 -a

  Found a working perl at 'C:\perl\bin\perl.exe'
  Traceback (most recent call last):
    File "build_ssl.py", line 253, in <module>
      main()
    File "build_ssl.py", line 187, in main
      os.chdir(ssl_dir)
  PermissionError: [Error 5] Access is denied: '..\\..\\openssl-1.0.1c'
  [55050 refs]