classification
Title: pyvenv pydoc.py script causing AttributeErrors on Windows
Type: behavior Stage: resolved
Components: Windows Versions: Python 3.4, Python 3.3
process
Status: closed Resolution: fixed
Dependencies: Superseder:
Assigned To: vinay.sajip Nosy List: eric.araujo, peter@psantoro.net, python-dev, vinay.sajip
Priority: normal Keywords:

Created on 2013-06-15 14:27 by peter@psantoro.net, last changed 2013-07-02 16:14 by eric.araujo. This issue is now closed.

Files
File name Uploaded Description Edit
shadow.py peter@psantoro.net, 2013-06-17 15:13 shadow.py test script
Messages (8)
msg191210 - (view) Author: Peter Santoro (peter@psantoro.net) * Date: 2013-06-15 14:27
I've recently hit an issue with pyvenv in Python 3.3.2 that is causing AttributeErrors in other packages on Windows (see https://groups.google.com/forum/?fromgroups#!topic/pylons-discuss/FpOSMDpdvy4).  Here's what I believe is going on:

On Windows, the pyvenv pydoc script has a .py extension - so import finds it instead of the system's pydoc module.  On Linux, the pyvenv pydoc script doesn't have an extension - so import finds the system's pydoc module.

I believe the Windows pyvenv pydoc.py script should be renamed to something like pydocs.py to prevent AttributeErrors.
msg191292 - (view) Author: Vinay Sajip (vinay.sajip) * (Python committer) Date: 2013-06-16 21:42
The pyvenv pydoc script should be in the venv's Scripts folder, which is not (normally) on sys.path - the venv's Lib\site-packages is. Can you provide a small script which demonstrates the problem while confirming that the venv's Scripts folder has not been added to sys.path?
msg191345 - (view) Author: Peter Santoro (peter@psantoro.net) * Date: 2013-06-17 15:13
As requested, I've attached a small test script called shadow.py.  Steps to reproduce:

1) pyvenv.py bugtest
2) copy the attached shadow.py script to bugtest and bugtest\scripts
3) cd bugtest
4) run shadow.py (first entry in sys.path is refers to bugtest directory per Python docs; finds the systems's pydoc module)
5) run bugtest\shadow.py (first entry in sys.path refers to bugtest\scripts directory per Python docs; finds the bugtest\scripts pydoc module instead of the system's pydoc module)

According to the Python documentation (http://docs.python.org/3/library/sys.html#sys.path):

"As initialized upon program startup, the first item of this list, path[0], is the directory containing the script that was used to invoke the Python interpreter. If the script directory is not available (e.g. if the interpreter is invoked interactively or if the script is read from standard input), path[0] is the empty string, which directs Python to search modules in the current directory first. Notice that the script directory is inserted before the entries inserted as a result of PYTHONPATH."

Maybe I'm missing something here, but isn't this problem caused by the fact that Python initializes sys.path[0] to contain the directory of the executing script and that having pydoc.py in that same directory (i.e. the venv's scripts directory) shadows the system's pydoc.py module?  On Linux, I didn't have this problem, because the pydoc script doesn't have the .py extension.  However, if you rename the pydoc script on Linux to pydoc.py, the same problem occurs.

I don't think a pydoc.py (or any other .py file which shadows a system module) can exist in the venv scripts (or bin) directory without shadowing/breaking the system provided module.  Maybe a pydoc.exe or pydoc.bat file is needed on Windows?  Another option would be to rename the pydoc.py file to something like pydocs.py, but that would be incompatible with other platforms and the existing documentation.
msg191348 - (view) Author: Vinay Sajip (vinay.sajip) * (Python committer) Date: 2013-06-17 16:03
> the fact that Python initializes sys.path[0] to contain the directory of the executing script

Of course, silly me. Sorry.

I would prefer to remove the pydoc script altogether. The other alternative would be to rename it to pydoc-script.py and have a pydoc.exe adjacent to it (this is how scripts installed with setuptools or distlib work). However, the simple launcher which would be pydoc.exe is not currently part of Python. I think the best thing would be for me to post on python-dev to see what views people there have about these alternatives.
msg191349 - (view) Author: Vinay Sajip (vinay.sajip) * (Python committer) Date: 2013-06-17 16:30
Rethinking, renaming seems more reasonable. I've posted about it to python-dev:

http://mail.python.org/pipermail/python-dev/2013-June/126904.html
msg191987 - (view) Author: Vinay Sajip (vinay.sajip) * (Python committer) Date: 2013-06-28 10:35
On further reflection: the pydoc script adds no value over and above "python -m pydoc <args>", so I think I will remove it.
msg192085 - (view) Author: Roundup Robot (python-dev) Date: 2013-06-30 21:12
New changeset af837bf390d0 by Vinay Sajip in branch '3.3':
Issue #18224: Removed pydoc script from created venv, as it causes problems on Windows and adds no value over and above python -m pydoc ...
http://hg.python.org/cpython/rev/af837bf390d0

New changeset de73e7ffabb3 by Vinay Sajip in branch 'default':
Closes #18224: Removed pydoc script from created venv, as it causes problems on Windows and adds no value over and above python -m pydoc ...
http://hg.python.org/cpython/rev/de73e7ffabb3

New changeset c17fa2cbad43 by Vinay Sajip in branch '3.3':
Issue #18224: Updated test.
http://hg.python.org/cpython/rev/c17fa2cbad43

New changeset ae69436eb7c2 by Vinay Sajip in branch 'default':
Issue #18224: Updated test.
http://hg.python.org/cpython/rev/ae69436eb7c2
msg192208 - (view) Author: Éric Araujo (eric.araujo) * (Python committer) Date: 2013-07-02 16:14
That’s too bad; it’s annoying to lose the ability to just use “pydoc x” if a venv is activated.
History
Date User Action Args
2013-07-02 16:14:06eric.araujosetnosy: + eric.araujo
messages: + msg192208
2013-06-30 21:12:10python-devsetstatus: open -> closed

nosy: + python-dev
messages: + msg192085

resolution: fixed
stage: resolved
2013-06-28 10:35:51vinay.sajipsetassignee: vinay.sajip
messages: + msg191987
versions: + Python 3.4
2013-06-17 16:30:00vinay.sajipsetmessages: + msg191349
2013-06-17 16:03:34vinay.sajipsetmessages: + msg191348
2013-06-17 15:13:19peter@psantoro.netsetfiles: + shadow.py

messages: + msg191345
2013-06-16 21:42:00vinay.sajipsetmessages: + msg191292
2013-06-15 18:34:36ned.deilysetnosy: + vinay.sajip
2013-06-15 14:27:13peter@psantoro.netcreate