classification
Title: default value for progname in pythonrun.c should be python3 for Python 3
Type: behavior Stage: resolved
Components: Versions: Python 3.2, Python 3.3
process
Status: closed Resolution: fixed
Dependencies: Superseder:
Assigned To: Nosy List: Arfrever, Joshua.Cogliati, benjamin.peterson, eric.araujo, georg.brandl, loewis, pitrou, python-dev
Priority: normal Keywords:

Created on 2012-06-06 22:37 by Joshua.Cogliati, last changed 2012-07-05 19:00 by pitrou. This issue is now closed.

Messages (12)
msg162425 - (view) Author: Joshua Cogliati (Joshua.Cogliati) Date: 2012-06-06 22:37
In Python/pythonrun.c the following definition exists:
static wchar_t *progname = L"python";

This is then used by Py_GetProgramName which is used by calculate_path in Modules/getpath.c 

Since in python 3, the default executable is python3, and not python, when calculate_path searches for "python" it will find where python 2 is installed instead of where python3 is installed.  This is the error message that this causes:

Could not find platform independent libraries <prefix>
Could not find platform dependent libraries <exec_prefix>
Consider setting $PYTHONHOME to <prefix>[:<exec_prefix>]

I recommend that line be changed to:
static wchar_t *progname = L"python3";

since that is the default executable name.

For my purposes (which is making an executable that embedds python 3), I was able to work around this by calling: 
Py_SetProgramName(L"python3");
before
Py_Initialize();
msg162426 - (view) Author: Joshua Cogliati (Joshua.Cogliati) Date: 2012-06-06 22:48
Here is a part of an strace where Python fails to find python3:
(This would work if progname=L"python3" )
...
23249 stat("/opt/python/3.2.2.3/bin/python", 0x7fff2881cbf0) = -1 ENOENT 
(No such file or directory)
23249 readlink("/usr/local/bin/python", "/usr/bin/python", 4096) = 15
23249 readlink("/usr/bin/python", "python2.7", 4096) = 9
23249 readlink("/usr/bin/python2.7", 0x7fff2881bb70, 4096) = -1 EINVAL (Invalid argument)
23249 stat("/usr/bin/Modules/Setup", 0x7fff2881cbf0) = -1 ENOENT (No such file or directory)
23249 stat("/usr/bin/lib/python3.2/os.py", 0x7fff2881cbf0) = -1 ENOENT (No such file or directory)
23249 stat("/usr/bin/lib/python3.2/os.pyc", 0x7fff2881cbf0) = -1 ENOENT (No such file or directory)
23249 stat("/usr/lib/python3.2/os.py", 0x7fff2881cbf0) = -1 ENOENT (No such file or directory)
23249 stat("/usr/lib/python3.2/os.pyc", 0x7fff2881cbf0) = -1 ENOENT (No such file or directory)
23249 stat("/lib/python3.2/os.py", 0x7fff2881cbf0) = -1 ENOENT (No such file or directory)
23249 stat("/lib/python3.2/os.pyc", 0x7fff2881cbf0) = -1 ENOENT (No such file or directory)
23249 write(2, "Could not find platform independ"..., 55) = 55
...
$ which python3
/opt/python/3.2.2.3/bin/python3
msg164548 - (view) Author: Éric Araujo (eric.araujo) * (Python committer) Date: 2012-07-02 20:52
Any opposition?
msg164549 - (view) Author: Éric Araujo (eric.araujo) * (Python committer) Date: 2012-07-02 20:52
Joshua: what command did you run under strace?
msg164553 - (view) Author: Antoine Pitrou (pitrou) * (Python committer) Date: 2012-07-02 21:30
This sounds reasonable. Is there any reason that this change might be detrimental?
msg164665 - (view) Author: Arfrever Frehtes Taifersar Arahesis (Arfrever) * Date: 2012-07-04 23:18
Maybe it would be better to use L"python3.2" for Python 3.2 and L"python3.3" for Python 3.3.
msg164686 - (view) Author: Joshua Cogliati (Joshua.Cogliati) Date: 2012-07-05 16:48
> Joshua: what command did you run under strace?

A program I created that embeds python3.  I could create a minimum piece of code that triggered the bug if needed.

> Maybe it would be better to use L"python3.2" for Python 3.2 and L"python3.3" for Python 3.3.

L"python3.2" and L"python3.3" would work better than the current L"python" since calculate_path assumes that progname is the name of an actual executable for python 3.  It only sorta works now with L"python" if python 2 and python 3 are installed in the same location, which was not the case for me since python 2 came with the system and python 3 was a local install.
msg164689 - (view) Author: Antoine Pitrou (pitrou) * (Python committer) Date: 2012-07-05 18:16
Hmm, actually, there is a potential problem. While "python3" is the official binary under POSIX, under Windows it is "python" (well, "python.exe").

Joshua, if you are embedding Python, why don't you simply call Py_SetPath to set the search path appropriately? Or is it not enough? (I've lost memory of the mazy details of how we calculate paths :-S).
msg164691 - (view) Author: Joshua Cogliati (Joshua.Cogliati) Date: 2012-07-05 18:53
> Joshua, if you are embedding Python, why don't you simply call Py_SetPath to set the search path appropriately? Or is it not enough? (I've lost memory of the mazy details of how we calculate paths :-S).

Setting Py_SetPath manually would basically require me to replicate the work done in Modules/getpath.c to figure out where the python libraries are.  I already set PYTHONPATH to get it to find my own modules.  (Note that there is a big difference between setting PYTHONPATH the environmental variable and calling Py_SetPath, Py_SetPath assumes that you are setting the python library module paths as well.)

The basic problem is that in function calculate_path (inside of Modules/getpath.c ) it has the following code:

char *_path = getenv("PATH");
...
wchar_t *prog = Py_GetProgramName();
...
 while (1) {
...
            joinpath(progpath, prog);
            if (isxfile(progpath))
                break;
...

which goes through the path and tries to find an executable with the same name as returned by Py_GetProgramName()

So if I do a """Py_SetProgramName(L"python3");""" that method works because prog="python3" but if I don't then the method fails because prog="python".  

Basically, to fix this bug, somehow, "wchar_t *prog =" in calculate_path needs to get the actual python executable for this version of python.
msg164692 - (view) Author: Antoine Pitrou (pitrou) * (Python committer) Date: 2012-07-05 18:55
Okay, that's convincing enough. Besides, I don't think it has ever worked for Windows, since it misses the adding of a ".exe" suffix.
msg164694 - (view) Author: Roundup Robot (python-dev) Date: 2012-07-05 18:59
New changeset c97d78415f5a by Antoine Pitrou in branch '3.2':
Issue #15020: The program name used to search for Python's path is now "python3" under Unix, not "python".
http://hg.python.org/cpython/rev/c97d78415f5a

New changeset 61e6ac40c816 by Antoine Pitrou in branch 'default':
Issue #15020: The program name used to search for Python's path is now python3 under Unix, not python.
http://hg.python.org/cpython/rev/61e6ac40c816
msg164695 - (view) Author: Antoine Pitrou (pitrou) * (Python committer) Date: 2012-07-05 19:00
This should be fixed now. Thanks!
History
Date User Action Args
2012-07-05 19:00:44pitrousetstatus: open -> closed
resolution: fixed
messages: + msg164695

stage: resolved
2012-07-05 18:59:29python-devsetnosy: + python-dev
messages: + msg164694
2012-07-05 18:55:28pitrousetmessages: + msg164692
2012-07-05 18:53:40Joshua.Cogliatisetmessages: + msg164691
2012-07-05 18:16:12pitrousetmessages: + msg164689
2012-07-05 16:48:18Joshua.Cogliatisetmessages: + msg164686
2012-07-04 23:18:20Arfreversetnosy: + Arfrever
messages: + msg164665
2012-07-02 21:30:36pitrousetnosy: + pitrou
messages: + msg164553
2012-07-02 20:52:43eric.araujosetmessages: + msg164549
2012-07-02 20:52:00eric.araujosetnosy: + georg.brandl, benjamin.peterson
messages: + msg164548
2012-06-13 14:51:06Joshua.Cogliatisettitle: Poor default value for progname in pythonrun.c -> default value for progname in pythonrun.c should be python3 for Python 3
2012-06-08 18:00:10eric.araujosetnosy: + eric.araujo

title: Poor default value for progname -> Poor default value for progname in pythonrun.c
2012-06-07 08:40:27pitrousetnosy: + loewis

versions: + Python 3.3
2012-06-06 22:48:25Joshua.Cogliatisetmessages: + msg162426
2012-06-06 22:37:28Joshua.Cogliaticreate