I tried to link a program against the libpython27.a provided by the latest 2.7.9 amd64 installer, only to discover that the provided library is a 32-bit version.  I had to go through the gendef/dlltool dance in order to produce a useable 64-bit library from the DLL.

Thanks for the notice. I'll try and get that fixed up for the next 2.7 release.

Is the libpython27.a file actually a 32-bit version or is it just corrupted? It seems to be considerably smaller than the version in the 32-bit installer, but it is certainly not being generated from the 32-bit version.

I didn't write this code originally, and I don't use MinGW, so I'm not entirely familiar with how it should work. Do I need separate 64-bit tools to build the library for the 64-bit DLL or will the 32-bit tools suffice? Are the interfaces stable enough within 64-bit versions of MinGW that shipping the library makes sense, or are people likely to always need to regenerate it anyway?

The libpython27.a is an actual 32-bit version, as confirmed by running objdump -t on it.  It reports the sections' file format as pe-i386 instead of pe-x86-64.  I am only using it for building for the 64-bit target, so I cannot confirm its viability for 32-bit builds.

To build 64-bit Windows binaries, you need a MinGW-w64 toolchain (which is a completely separate project from the original MinGW project).  The target triplet is x86_64-w64-mingw32.

I cannot speak to the stability of the MinGW-w64 library interface, but I would expect it to be stable.  To wit, I would be shocked if future changes required regeneration of third-party .a files.

So I've grabbed gendef and dlltool from the latest mingw-w64 and will look at using those in the future for both 2.7 and 3.5.

According to objdump, I can use these to create file format pe-i386 and pe-x86-64 with the same tools. Are these the correct formats for 32-bit and 64-bit respectively?

My commands are (approximately):

  gendef - python35.dll > mingwlib.def

  dlltool --dllname python35.dll --def mingwlib.def --output-lib amd64\libpython35.a -m i386:x86-64

  dlltool --dllname python35.dll --def mingwlib.def --output-lib win32\libpython35.a -m i386

Yes, pe-i386 and pe-x86-64 are the respective 32-bit and 64-bit object formats.

Your commands seem reasonable.  With gendef, I just let it create a .def file with the same name (i.e. skip the '-' and redirection); in my mind, that reinforces the association between the dll and def files.  With dlltool, I did not have to use the -m flag (as x86-64 is the default for me), but I don't see anything wrong with being explicit.

Just came across this advice on https://github.com/cython/cython/wiki/64BitCythonExtensionsOnWindows:

> Do not use MinGW-w64. As you will notice, the MinGW import library for
> Python (e.g. libpython27.a) is omitted from the AMD64 version of
> Python. This is deliberate. Do not try to make one using dlltool.
> There is no official MinGW-w64 release yet, it is still in "beta" and
> considered unstable, although you can get a 64-bit build from e.g.
> TDM-GCC. There have also been issues with the mingw runtime
> conflicting with the MSVC runtime; this can happen from places you
> don't expect, such as inside runtime libraries for g++ or gfortran. To
> stay on the safe side, avoid MinGW-w64 for now.

How accurate is this? Would we be better to omit the mingw libraries from the installer and instead provide the commands (or even a shell script?) to generate it with whatever toolset you're currently using?

That's certainly an interesting data point.  We are just beginning to use MinGW-w64 internally, so I do not have enough experience to confirm or deny that advice.  For various reasons, we must use cross-compiling on a Linux host, so the advice to use a native compiler is moot for our situation.

Certainly, documenting the absense of the 64-bit library would be good.  Providing a documented script to generate one is better.  Providing the library would be ideal, if there will not be forward compatibility or runtime issues.

It would be nice to see concrete details about the current state of affairs.  The cython project's warning would carry more weight with me if it contained links to specific details: mailing list discussion that led to the "deliberate" decision to omit the 64-bit library, bug reports filed against the mingw-w64 project about the "runtime issues", etc..  That said, such details probably do exist, but my cursory searching has failed to turn them up.

It looks like #1088716 is where the change was made a bit over 10 years ago. Adding Paul to see if he has any further insight, but I'm pretty sure at that point there was no 64-bit Python (there certainly wasn't a 64-bit Windows that people were using), and probably nobody noticed/cared that it wasn't there. It may even have been missing completely from the 64-bit installer until I took over building and started doing it slightly differently from Martin.

I've attached a patch for msi.py that will use mingw-w64's tools when building the installer, but I suspect we're better off putting the instructions there rather than hoping build versions match.

Is there a mingw list we can post this thread to for opinions from more people who depend on this lib?

There's a lot of politics around the mingw vs mingw64 situation, but in practice I believe the various mingw64 builds are fine. So I see no reason not to supply a correct 64-bit libpythonXY.a.

When the patch was made to include the libpythonXY.a file to the msi (wow! I'd forgotten I'd ever done a patch for the MSI build) there was only 32-bit systems, and there certainly wasnt't a 64-bit mingw.

The added patch seems like a reasonable approach.

New changeset 536a816e7232 by Steve Dower in branch '2.7':
Issue #23199: libpython27.a in amd64 release is 32-bit
https://hg.python.org/cpython/rev/536a816e7232

I've downloaded and installed python-2.7.9.amd.msi and spent some hours trying to understand the myriad of i386/x86-64 related linker errors I get when attempting to build an extension module. Now that I found this report I can at least stop doubting my sanity.

As I take it that there won't be an updated package any time soon, would someone knowledgeable please be so kind and place copy-paste ready step-by-step instructions here on what to do to get a working 64bit .a file for the 2.7.9 amd64 package? This would be much appreciated. Thanks.

I posted these commands above (modulo version number). If you have MinGW and binutils then it should Just Work, but I'm not very experienced with the range of installations you could have, so you may need to track down the gendef and dlltool tools:

  gendef - python27.dll > mingwlib.def

  dlltool --dllname python27.dll --def mingwlib.def --output-lib amd64\libpython27.a -m i386:x86-64

  dlltool --dllname python27.dll --def mingwlib.def --output-lib win32\libpython27.a -m i386

Steve: is this fixed and ready for next time?

Yep