This issue tracker has been migrated to GitHub, and is currently read-only.
For more information, see the GitHub FAQs in the Python's Developer Guide.

classification
Title: Multi-layered symlinks to python cause runtime error. sys.path is malformed.
Type: behavior Stage:
Components: Interpreter Core Versions: Python 2.7
process
Status: open Resolution:
Dependencies: Superseder:
Assigned To: Nosy List: Jason.Howlett, ncoghlan, terry.reedy
Priority: normal Keywords: patch

Created on 2011-10-15 05:14 by Jason.Howlett, last changed 2022-04-11 14:57 by admin.

Files
File name Uploaded Description Edit
symlink_path_fix.patch Jason.Howlett, 2011-10-15 22:05
Messages (4)
msg145582 - (view) Author: Jason Howlett (Jason.Howlett) Date: 2011-10-15 05:14
This error is observed in Python 2.7.2.

Steps to recreate problem:

1) Create directory ~/pytest.
2) Untar Python 2.7.2 sources into ~/pytest
3) configure, build, and install with --prefix=~/pytest/pyinstall
4) mkdir -p ~/pytest/other/bin
5) cd ~/pytest/other/bin
6) ln -s ../../pyinstall/bin/python
7) cd ~/pytest
8) ln -s other/bin bin
9) bin/python

This will result in the following exception:
Traceback (most recent call last):
  File "/home/jason/pytest/bin/../../pyinstall/lib/python2.7/site.py", line 564, in <module>
    main()
  File "/home/jason/pytest/bin/../../pyinstall/lib/python2.7/site.py", line 546, in main
    known_paths = addusersitepackages(known_paths)
  File "/home/jason/pytest/bin/../../pyinstall/lib/python2.7/site.py", line 279, in addusersitepackages
    user_site = getusersitepackages()
  File "/home/jason/pytest/bin/../../pyinstall/lib/python2.7/site.py", line 254, in getusersitepackages
    user_base = getuserbase() # this will also set USER_BASE
  File "/home/jason/pytest/bin/../../pyinstall/lib/python2.7/site.py", line 243, in getuserbase
    from sysconfig import get_config_var
ImportError: No module named sysconfig

Placing 'print sys.path' on line 242 of site.py prints out something like this:

['/home/jason/pyinstall/lib/python27.zip', '/home/jason/pyinstall/lib/python2.7', '/home/jason/pyinstall/lib/python2.7/plat-linux2', '/home/jason/pyinstall/lib/python2.7/lib-tk', '/home/jason/pyinstall/lib/python2.7/lib-old', '/home/jason/pyinstall/lib/python2.7/lib-dynload']

Notice that the root directory is /home/jason/pyinstall instead of the correct value /home/jason/pytest/pyinstall.

Now, on line 65 of site.py, add import sysconfig. For some reason this works at this point in site.py, but not down on line 243.

In sysconfig.py, at line 94, add the following:
print 'SYSCONFIG:', sys.exec_prefix, _EXEC_PREFIX

This will print the following:
SYSCONFIG: /home/jason/pytest/bin/../../pyinstall /home/jason/pyinstall

Line 93 of sysconfig.py is a call to normpath that collapses the path. The root of this problem is that the value of sys.exec_prefix is not correct. The value it has will work fine if ls'ing it in a shell as it handles the symlink /home/jason/pytest/bin properly. Path calculations internal to python, however, do not end up with the correct results.

I believe the fix to this problem is to modify Modules/getpath.c, starting at line 488, and instead of looking only for a symlink in the python executable, also look for a link in any of the path elements leading up to the executable. Detecting and expanding these out should result, in this example, with sys.exec_prefix being /home/jason/pytest/pyinstall and the correct behavior in sysconfig, site, etc.
msg145608 - (view) Author: Jason Howlett (Jason.Howlett) Date: 2011-10-15 22:05
Here is a proposed patch that solves the problem I described. Hopefully someone more familiar with Module/gethpath.c can make sure that it is ok and make improvements if needed.
msg146139 - (view) Author: Terry J. Reedy (terry.reedy) * (Python committer) Date: 2011-10-21 21:47
For the tracker, 'crash' == seqfault or equivalent, with no traceback.
Unwanted traceback is bad behavior.
Please test with 3.2 also if you can.
msg146158 - (view) Author: Jason Howlett (Jason.Howlett) Date: 2011-10-22 03:36
Confirmed. The problem exists in 3.2.2 also.
History
Date User Action Args
2022-04-11 14:57:22adminsetgithub: 57393
2013-02-01 21:54:27brett.cannonsetnosy: - brett.cannon
2011-10-22 03:36:08Jason.Howlettsetmessages: + msg146158
2011-10-21 21:47:05terry.reedysetnosy: + terry.reedy, brett.cannon, ncoghlan
type: crash -> behavior
messages: + msg146139
2011-10-15 22:05:40Jason.Howlettsetfiles: + symlink_path_fix.patch
keywords: + patch
messages: + msg145608

title: Multi-layered symlinks cause runtime error. sys.path is malformed. -> Multi-layered symlinks to python cause runtime error. sys.path is malformed.
2011-10-15 05:14:40Jason.Howlettcreate