This issue tracker has been migrated to GitHub, and is currently read-only.
For more information, see the GitHub FAQs in the Python's Developer Guide.

Title: sys.path issue if sys.prefix contains a colon
Type: behavior Stage:
Components: Interpreter Core Versions: Python 2.6
Status: closed Resolution: out of date
Dependencies: Superseder:
Assigned To: brett.cannon Nosy List: Mattie, arigo, brett.cannon, georg.brandl, loewis, ronaldoussoren
Priority: normal Keywords:

Created on 2006-06-16 11:22 by ronaldoussoren, last changed 2022-04-11 14:56 by admin. This issue is now closed.

Messages (9)
msg28813 - (view) Author: Ronald Oussoren (ronaldoussoren) * (Python committer) Date: 2006-06-16 11:22
If you install python in a directory whose path contains a colon sys.path 
will be wrong, the installation directory will be split into two strings and 
both of them are added to sys.path.

The following session demonstrates the problem:

bump:~/src/python/:colon bob$ ./python.exe -c "import sys; print 
'import site' failed; use -v for traceback
['', '/usr/local/lib/', '/Users/bob/src/python/', 'colon/../
Lib/', '/Users/bob/src/python/', 'colon/../Lib/plat-darwin', '/Users/bob/
src/python/', 'colon/../Lib/plat-mac', '/Users/bob/src/python/', 'colon/../
Lib/plat-mac/lib-scriptpackages', '/Users/bob/src/python/', 'colon/../
Lib/lib-tk', '/Users/bob/src/python/', 'colon/Modules']

msg28814 - (view) Author: Georg Brandl (georg.brandl) * (Python committer) Date: 2006-06-22 19:08
Logged In: YES 

I don't know if something can be done here. A possibility
would be, if one path segment doesn't exist, add the colon
and the next segment and try again. Martin, do you have an
msg28815 - (view) Author: Martin v. Löwis (loewis) * (Python committer) Date: 2006-06-24 10:35
Logged In: YES 

I think this can be fixed. sys.prefix should be treated as
unspittable when constructing sys.path, i.e. sys.prefix
should be inserted only after the colon-splitting has been done.

Not sure what the best way to implement that would be,
though, so unassigning.
msg28816 - (view) Author: Armin Rigo (arigo) * (Python committer) Date: 2006-06-25 10:04
Logged In: YES 

Internally, CPython sets up its path using a single
':'-separated string.  I have no clue how, on Posix, you are
supposed to use paths containing ':' in environment variables
like $PATH - escape the colons somehow?

If there is some escape mechanism, we could support it,
both as an internal way to fix the present issue, and
directly in parsing PYTHONPATH.  If there is no such
mechanism, then I can only conclude that having colons in
paths will make many other things unhappy, not just Python.
msg28817 - (view) Author: Martin v. Löwis (loewis) * (Python committer) Date: 2006-06-25 10:16
Logged In: YES 

The issue is not with environment variables at all; they
play no role in this bug report. There is no convention for
an escape mechanism in environment variables, and we should
not introduce one, but again, this report is not about
environment variables.
msg28818 - (view) Author: Ronald Oussoren (ronaldoussoren) * (Python committer) Date: 2006-06-25 21:07
Logged In: YES 

One way to fix this to redefine Py_GetPath to return some kind of array 
instead string that is formatted like PYTHONPATH. 

Another option is returning a string where path elements are seperated by 
NUL bytes (with two NUL bytes at the end to make it possible to find the end 
of the path).  A problem with this is that this would make it impossible to 
have empty strings in your PYTHONPATH (such as 'somedir::otherdir').

Both options have the disadvantage that they change a public interface 
(embedders can reimplement Py_GetPath to get full control over the 
construction of sys.path).

BTW. This bug was found by a user of py2app, a py2exe-like tool for macosx. 
A user of his application complained that the application didn't run when it 
was stored in a directory with colons in its name. 
msg59065 - (view) Author: Mattie (Mattie) Date: 2007-12-31 22:39
This is also proving to be a problem in various places for applications
embedding Python 2.5 on Linux. For example, my EventScripts plugin
embeds Python into Counter-Strike: Source game servers. It's not
uncommon at all for these servers to use a colon in their path names, as
rental companies will often assign the directories as
myuser/IPADDRESS:PORT/cstrike. For embedding Python we have some tricks
that allow us to bypass this sometimes, but it is rather inconvenient.

In addition to sys.path, I haven't looked into it deeply, but I believe
the codecs search path also seems to get confused when the local
directory has a colon somewhere in it.

Please consider revitalizing this bug, if possible. Thanks for
considering it.
msg85132 - (view) Author: Brett Cannon (brett.cannon) * (Python committer) Date: 2009-04-02 01:40
Autoconf will not even generate a Makefile that will run with a prefix
of ``/tmp/\:colon`` so closing as out of date since their is a more
fundamental toolchain issue outside of our control.
msg202853 - (view) Author: Ronald Oussoren (ronaldoussoren) * (Python committer) Date: 2013-11-14 15:46
AFAIK this still is a problem, although as Brett mentioned you have to do some work to end up with a non-functional install.

I ran into this in the context of py2app, that output of py2app basically contains a "portable" python installation and users can change the value of sys.prefix by dragging the application into a different folder. When applications are launched from a folder whose name contains a colon the application currently does not launch due to this problem.

I have no idea yet as to how to cleanly the py2app problem. I'll probably end up with a symlink hack for that ;-(

It should be easy enough to provide a patch for 3.5 that uses an array of path elements instead of the raw $PYTHONPATH value, but that requires changes to Python's startup code and I'll probably just try to contribute to Nick's plans to cleanup the APIs for interpreter configuration (PEP 432) instead of providing a patch here.
Date User Action Args
2022-04-11 14:56:18adminsetgithub: 43514
2013-11-14 15:46:13ronaldoussorensetmessages: + msg202853
2009-04-02 01:40:56brett.cannonsetstatus: open -> closed
resolution: out of date
messages: + msg85132
2009-02-11 03:05:39ajaksu2setassignee: brett.cannon
versions: + Python 2.6, - Python 2.5
type: behavior
nosy: + brett.cannon
2007-12-31 22:39:36Mattiesetnosy: + Mattie
messages: + msg59065
2006-06-16 11:22:57ronaldoussorencreate