classification
Title: python windows 2.7.8 64-bit did not install
Type: behavior Stage: resolved
Components: Installation, Windows Versions: Python 2.7
process
Status: open Resolution: works for me
Dependencies: Superseder:
Assigned To: Nosy List: Andreas.Richter, Henning.von.Bargen, loewis, terry.reedy, zach.ware
Priority: normal Keywords:

Created on 2014-08-05 11:32 by Andreas.Richter, last changed 2016-12-19 07:59 by Henning.von.Bargen.

Messages (20)
msg224821 - (view) Author: Andreas Richter (Andreas.Richter) Date: 2014-08-05 11:32
Having the following problem.
1. Download 2.7.8 64-bit windows msi
2. Install it
3. Run python ... says it's 2.7.6
4. Execute import hmac
5. Error

Output:
C:\prototype>python
Python 2.7.5 (default, May 15 2013, 22:44:16) [MSC v.1500 64 bit (AMD64)] on win32
Type "help", "copyright", "credits" or "license" for more information.
>>> import hmac
Traceback (most recent call last):
  File "<stdin>", line 1, in <module>
  File "C:\prototype\Tools\python27\lib\hmac.py", line 8, in <module>
    from operator import _compare_digest as compare_digest
ImportError: cannot import name _compare_digest

I saw some posts out there indicating it had something to do with django or something related, but when I run python -v I can see that it's only involving local python libraries (in fact no site-packages at all)
The problem is that the builtin version of the operator module (which incidentally is not loaded from a pyd or pyc file (meaning it's inside of either python27.dll or python.exe doesn't contain _compare_digest.)
Most of the time to get things to run you can just delete the import line if you can make sure not to call hmac.compare_digest.
msg224822 - (view) Author: Andreas Richter (Andreas.Richter) Date: 2014-08-05 11:34
Also note....
Even though I install 2.7.8 msi the python command line says that it's running 2.7.5
msg224836 - (view) Author: Martin v. Löwis (loewis) * (Python committer) Date: 2014-08-05 14:41
Can you please print sys.executable?
msg224841 - (view) Author: Zachary Ware (zach.ware) * (Python committer) Date: 2014-08-05 14:56
This seems very odd; I'm skeptical that this is a bug with Python or its installer.  A few questions to try to narrow things down:

1) Did you already have a version of Python installed, what version, and where?
2) Where did you install 2.7.8?
3) What output do you get from "where python"?
4) In addition to sys.executable that Martin asked for, what values do you have for sys.prefix and sys.path?
msg225084 - (view) Author: Terry J. Reedy (terry.reedy) * (Python committer) Date: 2014-08-08 19:30
I downloaded the 2.7.8 64-bit msi, installed over 2.7.7, and indeed got 2.7.8, which inports hmac fine.

Andreas, did you have 2.7.5/6 installed  before? If so, I strongly suspect that the installer did not run (correctly). There might be an error log somewhere, but the likely fix is to remove your current 2.7.z and re-try 2.7.8 install, possibly after re-downloading. I once had to do something similar and there have been similar reports on python-list.
msg225092 - (view) Author: Andreas Richter (Andreas.Richter) Date: 2014-08-09 00:58
Well I know I tried the installation and uninstallation multiple times. I also deleted the binaries and did a repair. Each time the old binaries came back. So I suppose there could be an old registered msi database be getting in the way. But since the non-binary files (like all the py files and libraries and so on) are of the right version I would doubt that. But I can check again. What steps should I do to "remove this bad source of binary files?"
msg225093 - (view) Author: Andreas Richter (Andreas.Richter) Date: 2014-08-09 01:10
Ok, complete uninstallation of 2.7.8 and then reinstalling did give me the new executable. In my use case I do not register extensions or add python to the path so I did not expect anything to be left over as "registration" of 2.7.5 previously, but I guess it was registered somewhere.
Thanks for checking, sorry. It might be nice to be able to get a zip file of the binaries instead of and msi file.
msg225275 - (view) Author: Andreas Richter (Andreas.Richter) Date: 2014-08-13 12:28
I think we can close this although there should be some kind of warning that running an upgrade without an uninstall may not upgrade the files in some cases. However I can't really tell what those cases are. Since I deleted all the files and then did a repair with the new installer and still the old dll came back, I can't tell where it came from. Especially since all the files in the lib folder were the newer ones. Originally I thought maybe the dll was locked during the install, but moving the whole installation folder and creating a new one and then doing a repair still recovered the old dll. I was able to get some more information on this, because we actually checking the python binaries into our git repo to make it easy for other users to immediately startup and go without the need to install various binaries on their machine.
msg225278 - (view) Author: Martin v. Löwis (loewis) * (Python committer) Date: 2014-08-13 13:40
Since we can't investigate further (since you have now resolved it for you), closing it as suggested.
msg226917 - (view) Author: Henning von Bargen (Henning.von.Bargen) Date: 2014-09-15 14:34
Perhaps this issue should be re-opened.

I stumbled across a similar issue today, and made some potentially useful observations.

On a machine with an existing c:\python27 directory (Python 2.7.5), I installed Python 2.7.8 with the "install just for me" option into a new directory c:\python278.

That's because there are several Python-based applications running; and I want to restrict the new Python version to one of them.

The application uses a different executable, LisaReportingAgent.exe.

Within the same environment (PATH first entry is c:\python278), calling python.exe, listing sys.path gives the expected output:

C:\Lisa\Kronos\reporting\test\bin>c:\python278\python
Python 2.7.8 (default, Jun 30 2014, 16:03:49) [MSC v.1500 32 bit (Intel)] on win
32
Type "help", "copyright", "credits" or "license" for more information.
>>> import sys
>>> for f in sys.path: print f
... 

C:\Lisa\Kronos\reporting\test
c:\python278\python27.zip
c:\python278\DLLs
c:\python278\lib
c:\python278\lib\plat-win
c:\python278\lib\lib-tk
c:\python278
c:\python278\lib\site-packages
c:\python278\lib\site-packages\win32
c:\python278\lib\site-packages\win32\lib
c:\python278\lib\site-packages\Pythonwin
>>> ^Z

Whereas the special executable's output is much different:

C:\Lisa\Kronos\reporting\test\bin>..\..\bin\win32-x86\LisaReportingAgent.exe
Python 2.7.5 (default, May 15 2013, 22:43:36) [MSC v.1500 32 bit (Intel)] on win
32
Type "help", "copyright", "credits" or "license" for more information.
>>> import sys
>>> for f in sys.path: print f
...

C:\Lisa\Kronos\reporting\test
C:\WINDOWS\system32\python27.zip
C:\Python278\Lib
C:\Python278\DLLs
C:\Python278\Lib\lib-tk
C:\Python27\Lib
C:\Python27\DLLs
C:\Python27\Lib\lib-tk
C:\Lisa\Kronos\reporting\test\bin
C:\Lisa\Kronos\reporting\bin\win32-x86
C:\Python27
C:\Python27\lib\site-packages
C:\Python27\lib\site-packages\win32
C:\Python27\lib\site-packages\win32\lib
C:\Python27\lib\site-packages\Pythonwin
>>>

Note the entries with c:\windows and with the different Python installation C:\python27. They are pointing to the old Python installation.

The executable is built from the following C code (except for initializing a SWIG-generated Python wrapper for a PDF library):

int main( int argc, char* argv[] ) {
	int rc = 0;

        /* ... setup a license key ... */

	Py_SetProgramName(argv[0]);
	Py_Initialize();
    
        /* ... init PDF library ... */

        rc = Py_Main(argc, argv);
	Py_Finalize();

	/* ... shut down PDF library ... */

	return rc;
}

I've read something that Python 2.7 behaves differently on Windows regarding installation and setup in comparison to 2.6. It seems that the new python.exe performs registry magic?
msg226918 - (view) Author: Henning von Bargen (Henning.von.Bargen) Date: 2014-09-15 14:46
The online help says: 

"""
When Python is hosted in another .exe (different directory, embedded via COM, etc), the “Python Home” will not be deduced, so the core path from the registry is used. Other “application paths” in the registry are always read.
"""

and

"""
If the environment variable PYTHONHOME is set, it is assumed as “Python Home”. Otherwise, the path of the main Python executable is used to locate a “landmark file” (Lib\os.py) to deduce the “Python Home”. If a Python home is found, the relevant sub-directories added to sys.path (Lib, plat-win, etc) are based on that folder. Otherwise, the core Python path is constructed from the PythonPath stored in the registry.
"""

So I set PYTHONHOME=c:\python278, then call LisaReportingServer.exe again. Now the output for sys.path looks OK to me, but "import hmac" still fails:

C:\Lisa\Kronos\reporting\test\bin>..\..\bin\win32-x86\LisaReportingAgent.exe
Python 2.7.5 (default, May 15 2013, 22:43:36) [MSC v.1500 32 bit (Intel)] on win
32
Type "help", "copyright", "credits" or "license" for more information.
>>> import sys
>>> for f in sys.path: print f
...

C:\Lisa\Kronos\reporting\test
C:\WINDOWS\system32\python27.zip
c:\python278\DLLs
c:\python278\lib
c:\python278\lib\plat-win
c:\python278\lib\lib-tk
C:\Lisa\Kronos\reporting\bin\win32-x86
c:\python278
c:\python278\lib\site-packages
c:\python278\lib\site-packages\win32
c:\python278\lib\site-packages\win32\lib
c:\python278\lib\site-packages\Pythonwin
>>> import hmac
Traceback (most recent call last):
  File "<stdin>", line 1, in <module>
  File "c:\python278\lib\hmac.py", line 8, in <module>
    from operator import _compare_digest as compare_digest
ImportError: cannot import name _compare_digest
>>>
msg226919 - (view) Author: Henning von Bargen (Henning.von.Bargen) Date: 2014-09-15 14:50
Sorry, of course it's NOT OK, because there's still C:\WINDOWS\system32\python27.zip as the second entry.

BTW the relevant environment variables:

Path=c:\python278;C:\WINDOWS\system32;C:\WINDOWS;C:\WINDOWS\System32\Wbem;c:\lisa\Kronos\reporting\lib\win32-x86\pdfprint
PYTHONHOME=c:\python278
PYTHONPATH=C:\Lisa\Kronos\reporting\test\bin\..
msg226920 - (view) Author: Henning von Bargen (Henning.von.Bargen) Date: 2014-09-15 15:08
I don't see how this sys.path is related to the online help section "3.3.3. Finding modules". Where does c:\windows\python27.zip come from? And why does Python have to install anything in the Windows directory at all?
msg226921 - (view) Author: Henning von Bargen (Henning.von.Bargen) Date: 2014-09-15 15:34
The only "solution" I could find was to give up running 2.7.8 next to 2.7.5, then uninstall 2.7.8, then install 2.7.8 with the "install for all users" option selected.
msg226924 - (view) Author: Martin v. Löwis (loewis) * (Python committer) Date: 2014-09-15 15:57
Henning: if you have installed Python 2.7.5 "for all users", then this behavior is expected. Python finds python27.dll in system32, so even if you install 2.7.8 "just for me", it will still pick up 2.7.5 from system32.

Installing different 2.7 versions on the same machine isn't really supported, although it should work if they are all installed "just for me". The operator module comes from python27.dll.
msg226925 - (view) Author: Henning von Bargen (Henning.von.Bargen) Date: 2014-09-15 16:36
> Installing different 2.7 versions on the same machine isn't really supported, although it should work if they are all installed "just for me".

Sigh... I've come to the same conclusion.

Nevertheless it seems to me that the actual behavior is different from the online documentation "3.3.3 Finding modules" at https://docs.python.org/2/using/windows.html
msg226940 - (view) Author: Martin v. Löwis (loewis) * (Python committer) Date: 2014-09-15 22:10
How so? Every single sentence in this section is absolutely correct. It just doesn't talk about finding builtin modules at all, only about modules found on the Python path (.py and .pyd files). Builtin modules don't have to be found, because they are, well, built in.
msg227740 - (view) Author: Henning von Bargen (Henning.von.Bargen) Date: 2014-09-27 22:05
Martin, while I technically understand your anwers, I have to say that from an ordinary developer's perspective, the behavior is actually *not* expected.

It may be expected for python-dev experts, but not for those who are just programming _with_ python.

It did cost me half a day to find out what's going on. Luckily, this happened on an internal development machine and not while installing the application on one of our customer's machine.

IIRC earlier versions of python did not install a python2*.* ZIP or DLL into %WINDIR%\System32.

It should at least be documented that installing two different 2.7.x releases on the same machine is not supported.

Furthermore, the technical difference between "install for all users" and "install just for me" is not clear.

As you said, the docs do not even mention that after an installation "for all users" the system uses %WINDIR%\System32\python2x.zip for searching libraries.

For a developer it is not obvious that hmac is built in while other standard libs (implemented in pure Python) are not.

BTW, if it was built-in (technically), then it could not be not found, as it happened to Andreas and me.
Fool-proof built-in would mean: statically linked with the executable, not in a DLL.

I recommend to keep things as is, but add a few sentences to the docs.
msg227751 - (view) Author: Martin v. Löwis (loewis) * (Python committer) Date: 2014-09-28 07:05
Reopening for somebody to look at; I'm not interested in Python 2.7 anymore.
msg283594 - (view) Author: Henning von Bargen (Henning.von.Bargen) Date: 2016-12-19 07:59
A belated update... What I'm trying is similar to virtualenv:

My application should not depend on any pre-installed Python and not have any influence on it (and it should not modify the registry or the system environment variables).

It should be noted that my application is using a different executable instead of python.exe.

On Linux, I can do "make altinstall", but on Windows, setting up everything for compiling Python is out of scope.

Some of my other programs are using Python 3.5. With its "embeddable zip" it is very easy to achieve this goal.

Unfortunately, the most important program is using Python 2 on Windows and re-writing it for Python 3 would be quite an effort.

I think I finally found a way now with Python 2.7:

1. Define a directory where Python 2.x should be installed.
   SET PYTHONHOME=....
2. Create that directory
   MKDIR %PYTHONHOME%
3. Run the installer like this:
   msiexec /a python-2.7.12.amd64 /qb TARGETDIR=%PYTHONHOME%
4. Useful: pushd %PYTHONHOME%; python -m ensurepip; popd
5. Copy python 27.dll next to your executable (see http://blog.pyspoken.com/2016/04/27/embedding-python-how-to-confuse-python-and-yourself) - this is the important step.
   copy /b %PYTHONHOME%\python27.dll %MY_APPLICATION_HOME%\bin\
6. When running your executable, use a script which sets %PYTHONHOME%
   copy /b %PYTHONHOME%\
History
Date User Action Args
2016-12-19 07:59:09Henning.von.Bargensetmessages: + msg283594
2014-09-28 07:05:27loewissetstatus: closed -> open

messages: + msg227751
2014-09-27 22:05:15Henning.von.Bargensetmessages: + msg227740
2014-09-15 22:10:20loewissetmessages: + msg226940
2014-09-15 16:36:35Henning.von.Bargensetmessages: + msg226925
2014-09-15 15:57:30loewissetmessages: + msg226924
2014-09-15 15:34:48Henning.von.Bargensetmessages: + msg226921
2014-09-15 15:08:52Henning.von.Bargensetmessages: + msg226920
2014-09-15 14:50:13Henning.von.Bargensetmessages: + msg226919
2014-09-15 14:46:12Henning.von.Bargensetmessages: + msg226918
2014-09-15 14:34:39Henning.von.Bargensetnosy: + Henning.von.Bargen
messages: + msg226917
2014-08-13 13:43:23zach.waresettype: crash -> behavior
components: + Windows
stage: resolved
2014-08-13 13:40:49loewissetstatus: open -> closed
resolution: works for me
messages: + msg225278
2014-08-13 12:28:58Andreas.Richtersetmessages: + msg225275
2014-08-09 01:10:23Andreas.Richtersetmessages: + msg225093
2014-08-09 00:58:53Andreas.Richtersetmessages: + msg225092
2014-08-08 19:30:46terry.reedysetnosy: + terry.reedy

messages: + msg225084
title: python windows 2.7.8 64-bit wrong binary version -> python windows 2.7.8 64-bit did not install
2014-08-05 14:56:29zach.waresetnosy: + zach.ware
messages: + msg224841
2014-08-05 14:41:04loewissetnosy: + loewis
messages: + msg224836
2014-08-05 11:34:21Andreas.Richtersetmessages: + msg224822
2014-08-05 11:32:54Andreas.Richtercreate