classification
Title: Distutils-based installer does not detect 64bit versions of Python
Type: behavior Stage: needs patch
Components: Distutils Versions: Python 3.3, Python 3.1, Python 2.7
process
Status: open Resolution:
Dependencies: Superseder:
Assigned To: tarek Nosy List: ChrisBenson, Eli_B, Roy.Jacobson, Stephen.White, brian.curtin, carwyn, cgohlke, dontbugme, epu, eric.araujo, erluk, ghazel, jason.coombs, jerome.radix, jkloth, loewis, menekali@gmail.com, pje, r.david.murray, santa4nt, sorin, srid, tarek, vr.gamer
Priority: normal Keywords:

Created on 2009-08-28 08:52 by erluk, last changed 2013-09-20 11:26 by ChrisBenson.

Messages (22)
msg92028 - (view) Author: (erluk) Date: 2009-08-28 08:52
An installer for source-only modules created using distutils
(bdist_wininst) will only detect 32bit installations of Python on 64bit
Windows machines.

Expected behaviour:
The installer lists 32bit and 64bit installations of Python on the machine.
msg92045 - (view) Author: Martin v. Löwis (loewis) * (Python committer) Date: 2009-08-28 21:34
Unfortunately, I don't think this is possible. When creating the
installer, you have to make a choice whether to make it a 32-bit or a
64-bit MSI file - whether or not you have any architecture dependency.

At least, can't think of an easy way to solve it.
msg96921 - (view) Author: Carwyn Edwards (carwyn) Date: 2009-12-27 20:42
This prevents numerous packages from installing correctly including the
current 0.6c11 version of setuptools. When the installer runs it reports
that it can't find the version of python installed from
python-2.6.4.amd64.msi.

It seems to be looking for it in:

[HKEY_LOCAL_MACHINE\SOFTWARE\Wow6432Node\Python\PythonCore\2.6\InstallPath]
@="C:\\Python26\\"

.. rather than the:
[HKEY_LOCAL_MACHINE\SOFTWARE\Python\PythonCore\2.6\InstallPath]
@="C:\\Python26\\"

.. that the python installer puts in the registry.

This is related to the conversation in this thread:

http://www.mail-archive.com/distutils-sig@python.org/msg10512.html

And to quote from http://bugs.python.org/setuptools/issue2 :

"The issue is with the .exe header used by the bdist_wininst command;
setuptools doesn't generate this wrapper itself; it uses the one
supplied by the distutils, so non-setuptools packages will have the same
problem.  Without a fixed .exe header in the Python installation, *all*
bdist_wininst-generated .exe installers (setuptools or distutils) will
have the same problem."
msg96925 - (view) Author: Martin v. Löwis (loewis) * (Python committer) Date: 2009-12-27 21:22
Carwyn: those packages just need to create two versions of their
installers - one for 32-bit Python, and one for 64-bit Python. Please
report that to the respective packages.
msg96928 - (view) Author: sorin (sorin) Date: 2009-12-27 22:12
It is possible to create combined x86 and x64 msi files and in fact it
would be a good idea to have only one instead of two.
msg96930 - (view) Author: Martin v. Löwis (loewis) * (Python committer) Date: 2009-12-27 23:03
sorin: can you provide a patch?
msg96933 - (view) Author: PJ Eby (pje) * (Python committer) Date: 2009-12-28 01:56
ISTM there may be two ways to fix this problem; one was to change the
.exe header produced by bdist_wininst, but in retrospect, it can't fix
this because it's likely Windows' 64-to-32 bit conversion (Wow6432Node)
that's changing the registry path used, rather than anything that Python
or distutils is doing.

The other way, which has more probability of working, is for the 64-bit
Python installer to include an extra registry entry that would allow the
32 bit versions to find the 64-bit version.

(This might also require an install-time 64/32 compatibility check being
added to the .exe header for bdist_wininst to prevent installing
binary-incompatible extensions.)
msg100396 - (view) Author: Stephen White (Stephen.White) Date: 2010-03-04 16:29
32bit apps can query the 64bit registry, using the appropriate security and access rights options such as KEY_WOW64_64KEY (0x0100).

Similarly KEY_WOW64_32KEY can be used for 64bit apps to read/write the 32bit registry without having to have knowledge of how the Wow6432Nodes are arranged .

These mean that a 64bit aware app, whether compiled as 64 or 32 bits, can access the alternative view of the registry.

http://msdn.microsoft.com/en-us/library/ms724897(VS.85).aspx
http://msdn.microsoft.com/en-us/library/ms724878(VS.85).aspx

For example if you have both 64 and 32 bit copies of Python installed then a Python app running under the 32bit copy of Python can query the location of the 64bit copy of Python using code like:
key64 = _winreg.OpenKey(_winreg.HKEY_LOCAL_MACHINE, "Software\\Python\\PythonCore\\2.6\\PythonPath", 0, _winreg.KEY_READ + 0x0100)
_winreg.QueryValue(key, "")

C code can do similarly.
msg101094 - (view) Author: Martin v. Löwis (loewis) * (Python committer) Date: 2010-03-15 11:00
One issue to consider is pre/post-install actions. bdist_wininst loads pythonxy.dll from the target system, which would fail if it is a 32-bit installer process that tries to load a 64-bit python DLL.
msg112936 - (view) Author: Santoso Wijaya (santa4nt) * Date: 2010-08-05 00:55
Does anyone know of any workaround, for now?
msg116126 - (view) Author: Éric Araujo (eric.araujo) * (Python committer) Date: 2010-09-11 22:34
Removing 2.6 which only gets security fixes now.
msg117899 - (view) Author: (dontbugme) Date: 2010-10-02 21:33
you can add  InstallPath key with the corresponding value at [HKEY_LOCAL_MACHINE\SOFTWARE\Wow6432Node\Python\PythonCore\2.6\]
if you want disutils installer to detect your python
That makes him detect and install the librarys or scripts to the right directory, but doens't make your library 64bit compatible if it isn't
(means if the library doesn't work on 64 bit i neither will whith this work around) Only possibility of fixing that problem is installing 32bit python
msg125977 - (view) Author: Francis Cullen (menekali@gmail.com) Date: 2011-01-11 01:25
There are many updated installers, for many libs for those of us using 64bit windows 7.

http://www.lfd.uci.edu/~gohlke/pythonlibs/
msg127618 - (view) Author: Jérôme Radix (jerome.radix) Date: 2011-01-31 16:56
Could you please change the priority of this Issue to 'High' as this problem is a big annoyance for all Windows 64bits users which is a rather large niche of users (a niche getting larger and larger everyday), don't you think ?
msg127627 - (view) Author: R. David Murray (r.david.murray) * (Python committer) Date: 2011-01-31 17:29
I'll leave priority setting to tarek, but it doesn't look to me like raising the priority is going to make any difference, since it doesn't sound from reading the ticket like anyone has found a solution yet (other than offering 64bit installers).
msg127630 - (view) Author: Tarek Ziadé (tarek) * (Python committer) Date: 2011-01-31 17:34
Yeah I agree. Until we get a solution + patch the priority here does not really matter.
msg151778 - (view) Author: vr gamer (vr.gamer) Date: 2012-01-22 17:05
I'm not certain that I agree with the argument used to justify keeping this as a 'normal' priority issue. Apparently, since it doesn't effect the entire python community and being as there is no readily available solution, the decision is to treat it as a minor problem. Were one to apply the same logic to, what say, epidemiology, then I suppose the lack of a vaccine for HIV would not be a particularly pressing issue either. 

If priority escalation is out of the question, can we at least have an update? After three years of workarounds, I'm beginning to suspect that the idea is to wait until the effected packages become deprecated and then declare 'solved!'.
msg151827 - (view) Author: Brian Curtin (brian.curtin) * (Python committer) Date: 2012-01-23 17:00
Without a patch or a solution, the priority doesn't really matter (like Tarek said in msg127630). If anyone is actively working on this feel free to say otherwise, but I see no status to update.
msg151851 - (view) Author: Martin v. Löwis (loewis) * (Python committer) Date: 2012-01-23 21:55
> If priority escalation is out of the question

It's not out of the question - it's just that setting the priority on
the issue is not a proper way to escalate.

Instead, there are two forms of escalation available:
1. submit a patch that fixes the issue
2. pay somebody to fix the issue if you can't fix it yourself
msg177389 - (view) Author: Roy Jacobson (Roy.Jacobson) Date: 2012-12-12 21:23
This bug is a really annoying one, any chance it will be fixed in 2.7? It's really a matter when you want to deploy a program using distutils (my case), because you cannot really require your clients to edit the registry themselves :/

Is there any problem with just adding the x32 compatibility path entry to the python x64 .msi? It's a very easy fix that shouldn't cause any harm.
msg184357 - (view) Author: Erik Purins (epu) Date: 2013-03-16 22:06
I would like to investigate this issue, but I need more information regarding the bug and the expected behavior. Is this specifically that an x64 windows python that generates a bdist (msi output) runs and can't find the x64 interpreter because of syswow registry redirection? That is, packaging should be able to find the interpreter bitness that generated the msi in the first place (and no-other bitness)?

There are python sprints this week at PyCon, but I cannot attend them. Clarifying the expected behavior this week will help me write tests and investigate/fix (if it is in my ability).
msg184358 - (view) Author: Martin v. Löwis (loewis) * (Python committer) Date: 2013-03-16 22:26
Erik: the issue is about bdist_wininst, not bdist_msi (bdist_msi has a similar issue, but it is entirely different in its causes and potential resolution, and shall not be discussed here).

The code to find the installations is in PC/bdist_wininst/install.c:GetPythonVersions. The code to run the installscript is in run_installscript.
History
Date User Action Args
2013-09-20 11:26:01ChrisBensonsetnosy: + ChrisBenson
2013-03-16 22:26:11loewissetmessages: + msg184358
2013-03-16 22:06:18epusetmessages: + msg184357
2013-03-16 22:05:41epusetnosy: + epu
2012-12-12 21:23:24Roy.Jacobsonsetnosy: + Roy.Jacobson
messages: + msg177389
2012-01-23 21:55:17loewissetmessages: + msg151851
2012-01-23 17:00:19brian.curtinsetmessages: + msg151827
2012-01-22 17:05:55vr.gamersetnosy: + vr.gamer
messages: + msg151778
2011-08-03 20:40:13Eli_Bsetnosy: + Eli_B
2011-04-04 22:41:37jason.coombssetnosy: + jason.coombs
2011-01-31 17:34:29tareksetnosy: loewis, pje, ghazel, tarek, jkloth, eric.araujo, dontbugme, r.david.murray, carwyn, sorin, brian.curtin, srid, cgohlke, erluk, Stephen.White, santa4nt, menekali@gmail.com, jerome.radix
messages: + msg127630
2011-01-31 17:29:51r.david.murraysetversions: + Python 3.3, - Python 3.2
nosy: + r.david.murray

messages: + msg127627

stage: needs patch
2011-01-31 16:56:50jerome.radixsetnosy: loewis, pje, ghazel, tarek, jkloth, eric.araujo, dontbugme, carwyn, sorin, brian.curtin, srid, cgohlke, erluk, Stephen.White, santa4nt, menekali@gmail.com, jerome.radix
messages: + msg127618
2011-01-31 16:00:32jerome.radixsetnosy: + jerome.radix
2011-01-11 08:22:21cgohlkesetnosy: + cgohlke
2011-01-11 01:25:24menekali@gmail.comsetnosy: + menekali@gmail.com
messages: + msg125977
2010-10-03 00:04:00jklothsetnosy: + jkloth
2010-10-02 21:36:53loewissetversions: - Python 2.6
2010-10-02 21:33:38dontbugmesetnosy: + dontbugme

messages: + msg117899
versions: + Python 2.6
2010-09-11 22:34:07eric.araujosetnosy: + eric.araujo

messages: + msg116126
versions: + Python 3.1, - Python 2.6
2010-09-11 14:13:16r.david.murraylinkissue9830 superseder
2010-08-05 00:55:00santa4ntsetnosy: + santa4nt
messages: + msg112936
2010-07-19 22:42:17ghazelsetnosy: + ghazel
2010-03-15 11:00:14loewissetmessages: + msg101094
2010-03-04 16:38:13brian.curtinsetnosy: + brian.curtin
2010-03-04 16:29:45Stephen.Whitesetnosy: + Stephen.White
messages: + msg100396
2009-12-28 01:56:13pjesetnosy: + pje
messages: + msg96933
2009-12-27 23:11:37tareksetversions: + Python 2.7, Python 3.2, - Python 2.5
2009-12-27 23:03:42loewissetmessages: + msg96930
2009-12-27 22:12:47sorinsetnosy: + sorin
messages: + msg96928
2009-12-27 21:22:44loewissetmessages: + msg96925
2009-12-27 20:53:32sridsetnosy: + srid
2009-12-27 20:42:33carwynsetnosy: + carwyn
messages: + msg96921
2009-08-28 21:34:31loewissetnosy: + loewis
messages: + msg92045
2009-08-28 08:52:18erlukcreate