classification
Title: When setting complete PYTHONPATH on Python 3.x, paths in the PYTHONPATH are ignored
Type: behavior Stage:
Components: Interpreter Core, Windows Versions: Python 3.0, Python 3.1
process
Status: closed Resolution: fixed
Dependencies: Superseder:
Assigned To: amaury.forgeotdarc Nosy List: Suzumizaki, amaury.forgeotdarc, davidsarah, fabioz, loewis
Priority: critical Keywords: patch

Created on 2009-05-04 19:06 by fabioz, last changed 2011-01-11 22:51 by loewis. This issue is now closed.

Files
File name Uploaded Description Edit
ProjectStructure.zip fabioz, 2009-05-04 19:06 Project used to reproduce the problem
large_pythonpath.patch amaury.forgeotdarc, 2009-06-09 09:34
Messages (8)
msg87151 - (view) Author: Fabio Zadrozny (fabioz) * Date: 2009-05-04 19:06
Setting the complete PYTHONPATH in Python 3.x does not work (reported
initially for Pydev:
https://sourceforge.net/tracker/index.php?func=detail&aid=2767830&group_id=85796&atid=577329
). Checked on 3.0.1 and 3.1a2.

I'm not sure if the paths must be the same, but when I did change the
path of the project, the bug was not always reproduced.

I've reproduced it on Windows XP SP3.

To reproduce, extract the ProjectStructure so that you have a path as
c:\Documents and
Settings\Fabio\Desktop\temp_pydev\ProjectStructure\proj_b\src\testcaller_a.py

Then considering your python install is at d:\bin\Python301, set the
pythonpath as:

[C:\temp]set pythonpath=C:\Documents and
Settings\Fabio\Desktop\temp_pydev\ProjectStructure\proj_b\src;C:\Documents
and
Settings\Fabio\Desktop\temp_pydev\ProjectStructure\proj_a\src;D:\bin\Python301;D:\bin\Python301\DLLs;D:\bin\Python301\lib;D:\bin\Python301\lib\plat-win;D:\bin\Python301\lib\site-packages

And run the command below:

[C:\temp]d:\bin\Python301\python.exe "c:\Documents and
Settings\Fabio\Desktop\temp_pydev\ProjectStructure\proj_b\src\testcaller_a.py"
Running code is 'c:\Documents and
Settings\Fabio\Desktop\temp_pydev\ProjectStructure\proj_b\src\testcaller_a.py'

The contents of 'sys.path' is:
c:\Documents and
Settings\Fabio\Desktop\temp_pydev\ProjectStructure\proj_b\src
C:\WINDOWS\system32\python30.zip
D:\bin\Python301\DLLs
D:\bin\Python301\lib
D:\bin\Python301\lib\plat-win
D:\bin\Python301
D:\bin\Python301\lib\site-packages
Traceback (most recent call last):
  File "c:\Documents and
Settings\Fabio\Desktop\temp_pydev\ProjectStructure\proj_b\src\testcaller_a.py",
line 11, in <module>
    from testmod_a import testfunc
ImportError: No module named testmod_a


Note how the module was not properly imported... Now, if the PYTHONPATH
is set without the system folders:

[C:\temp]set pythonpath=C:\Documents and
Settings\Fabio\Desktop\temp_pydev\ProjectStructure\proj_b\src;C:\Documents
and Settings\Fabio\Desktop\temp_pydev\ProjectStructure\proj_a\src

Running the command line:

[C:\temp]d:\bin\Python301\python.exe "c:\Documents and
Settings\Fabio\Desktop\temp_pydev\ProjectStructure\proj_b\src\testcaller_a.py"
Running code is 'c:\Documents and
Settings\Fabio\Desktop\temp_pydev\ProjectStructure\proj_b\src\testcaller_a.py'

The contents of 'sys.path' is:
c:\Documents and
Settings\Fabio\Desktop\temp_pydev\ProjectStructure\proj_b\src
C:\Documents and
Settings\Fabio\Desktop\temp_pydev\ProjectStructure\proj_b\src
C:\Documents and
Settings\Fabio\Desktop\temp_pydev\ProjectStructure\proj_a\src
C:\WINDOWS\system32\python30.zip
D:\bin\Python301\DLLs
D:\bin\Python301\lib
D:\bin\Python301\lib\plat-win
D:\bin\Python301
D:\bin\Python301\lib\site-packages

'test_print()' in "testmod_a.testfunc" is called.

Properly works. Note that on Python 2.x this works.
msg87412 - (view) Author: Amaury Forgeot d'Arc (amaury.forgeotdarc) * (Python committer) Date: 2009-05-07 23:28
After some attempts, it seems that PYTHONPATH is completely ignored if 
it is longer than 256 characters
msg87413 - (view) Author: Amaury Forgeot d'Arc (amaury.forgeotdarc) * (Python committer) Date: 2009-05-07 23:39
Confirmed in PC/getpathp.c:
		size_t r = mbstowcs(wenvpath, _envpath, MAXPATHLEN+1);
		envpath = wenvpath;
		if (r == (size_t)-1 || r >= MAXPATHLEN)
			envpath = NULL;
msg89142 - (view) Author: Amaury Forgeot d'Arc (amaury.forgeotdarc) * (Python committer) Date: 2009-06-09 09:34
Here is a patch: fix+test.

I left the initial code, #ifdef'd for PC platforms different than
MS_WINDOWS. Just in case someone wants to port py3k to OS/2...
msg89166 - (view) Author: Martin v. Löwis (loewis) * (Python committer) Date: 2009-06-09 20:46
The patch looks right to me, please apply.
msg89169 - (view) Author: Amaury Forgeot d'Arc (amaury.forgeotdarc) * (Python committer) Date: 2009-06-09 21:32
Fixed in r73322 (py3k) and r73323 (3.0)
msg125948 - (view) Author: David-Sarah Hopwood (davidsarah) Date: 2011-01-10 22:51
Looking at http://svn.python.org/view/python/branches/py3k/PC/getpathp.c?r1=73322&r2=73321&pathrev=73322 , wouldn't it be better to add a Py_WGETENV function? There are likely to be other cases where that would be the correct thing to use.
msg126046 - (view) Author: Martin v. Löwis (loewis) * (Python committer) Date: 2011-01-11 22:51
Unless you think the code is actually incorrect as it stands, it's certainly worse to change it (in whatever respect) than to leave it.
History
Date User Action Args
2011-01-11 22:51:50loewissetnosy: loewis, amaury.forgeotdarc, fabioz, Suzumizaki, davidsarah
messages: + msg126046
2011-01-10 22:51:56davidsarahsetnosy: + davidsarah
messages: + msg125948
2009-06-09 21:32:37amaury.forgeotdarcsetstatus: open -> closed
resolution: accepted -> fixed
2009-06-09 21:32:24amaury.forgeotdarcsetmessages: + msg89169
2009-06-09 20:46:11loewissetassignee: amaury.forgeotdarc
resolution: accepted
messages: + msg89166
2009-06-09 09:34:23amaury.forgeotdarcsetfiles: + large_pythonpath.patch
keywords: + patch
messages: + msg89142
2009-05-08 01:55:21Suzumizakisetnosy: + Suzumizaki
2009-05-07 23:39:36amaury.forgeotdarcsetpriority: critical
nosy: + loewis
messages: + msg87413

2009-05-07 23:28:02amaury.forgeotdarcsetnosy: + amaury.forgeotdarc
messages: + msg87412
2009-05-04 19:06:09fabiozcreate